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Abstract 

In  order  to  bring  the  doctrine  of  Effects-Based  Operations  (EBO)  into  a  fully 
operational  capability,  Effects-Based  Assessment  (EBA)  must  provide  relevant  insight  to 
the  commander  and  his  planning  staff.  Assessments  of  an  effects-based  plan  and 
execution  must  include  an  assessment  of  the  effects  of  a  campaign  on  the  enemy  in 
addition  to  an  assessment  of  the  accomplishment  of  friendly  actions  taken  to  achieve  the 
desired  effects.  Detennining  the  effects  of  a  campaign  requires  an  analysis  of  the 
dynamics  of  the  enemy  systems.  EBA  must  be  able  to  recognize  the  states  of  the 
enemy’s  systems  as  the  system  states  change  over  time.  This  research  advances  the 
application  of  EBA  by  defining  anticipated  states  of  enemy  systems,  developing 
indicators  to  determine  those  states,  and  applying  progress  functions  to  the  states  in  order 
to  quantify  attainment  of  the  commander’s  objectives.  The  methodology  describes  a 
process  for  assessing  combat  and  stability  operations.  The  results  indicate  that  the  EBA 
methodology  developed  in  this  research  works  best  where  the  systems  of  interest  cannot 
be  assessed  directly. 
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A  METHODOLOGY  FOR  PERFORMING  EFFECTS-BASED  ASSESSMENTS 


1  Introduction 


1.1  Background 

In  his  2005  Phalanx  article,  “Re-Operationalizing  Analysis  for  the  Warfighter,” 
Air  Force  Chief  of  Staff  and  former  Combined  Forces  Air  Component  Commander 
(CFACC)  for  Operations  ENDURING  FREEDOM  and  IRAQI  FREEDOM,  General  T. 
Michael  Moseley  emphasizes  the  importance  of  an  operations  assessment  capability  for 
support  to  the  CFACC:  “We  need  to  focus  on  whether  we  are  creating  the  effect  we  want 
.  .  .  We  must  also  ensure  the  understanding  of  the  2nd  and  3rd  order  effects  we  are  having 
on  the  battlespace  and  our  enemies  ...”  (Moseley,  2005:8,  9).  The  Chiefs  statements 
embody  the  essence  of  Effects-Based  Operations  (EBO)  and  the  necessity  for 
commanders,  strategists,  planners,  and  assessors  to  gain  useful  knowledge  of  the  effects 
military  actions  achieve  during  combat  operations. 

As  a  CFACC,  General  Moseley  saw  first-hand  the  challenges  of  executing  EBO 
and  the  necessity  of  a  robust  assessment  capability.  Across  the  Air  Force,  the  operational 
and  analytical  communities  alike  have  developed  new  doctrine,  technologies,  and 
processes  to  meet  these  challenges  and  bring  the  EBO  concept  into  a  fully  operational 
capability.  The  thesis  develops  a  new  methodology  for  performing  Effects-Based 
Assessment  that  incorporates  the  lessons  learned  from  past  assessment  efforts  and 
concepts  from  recent  doctrine,  technology,  and  analytical  methods. 

Chapter  1  summarizes  the  definition  of  EBO  and  presents  how  the  assessments 
team  under  General  Moseley  implemented  an  effects-based  assessment  effort  in 
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Operation  IRAQI  FREEDOM  and  the  challenges  the  assessment  team  encountered. 

These  challenges  illustrate  well  the  problems  with  the  current  assessment  methodology. 
The  chapter  continues  with  a  detailed  problem  statement  and  the  objectives  of  this  thesis. 
The  chapter  concludes  with  an  overview  of  the  rest  of  the  document. 

1.1.1  Effects-Based  Operations  Defined. 

Several  definitions  for  EBO  exist  and  many  differ  only  in  their  jargon.  The 

United  States  Joint  Forces  Command  Joint  Warfighting  Center  Joint  Doctrine  Series 

Pamphlet  7  (JWFC  Pam  7)  offers  the  most  comprehensive  definition  of  EBO: 

Operations  that  are  planned,  executed,  assessed,  and  adapted  based  on  a 
holistic  understanding  of  the  operational  environment  in  order  to  influence 
or  change  system  behavior  or  capabilities  using  the  integrated  application 
of  selected  instruments  of  power  to  achieve  directed  policy  aims.  (JWFC 
Doctrine  Pam  7,  2004:2) 

JWFC  Pam  7  extends  this  definition  by  decomposing  EBO  into  four  major  components: 
System-of-Systems  Analysis,  Effects-Based  Planning,  Effects-Based  Execution,  and 
Effects-Based  Assessment. 

A  System-of-Systems  Analysis  (SoSA)  model  provides  the  holistic  understanding 
of  the  operational  environment  by  characterizing  the  complex  organization,  relationships, 
and  key  characteristics  of  an  enemy  political,  military,  economic,  social,  information,  and 
infrastructure  (PMESII)  network.  The  SoSA  recognizes  the  multifarious  linkages  across 
the  enemy  PMESII  system  and  describes  the  system  using  network  topology.  In  the 
SoSA  model,  the  key  components  of  the  system  are  represented  by  nodes,  and  the 
connections  between  the  components  are  represented  by  arcs,  as  shown  in  Figure  1.1 
(JWFC  Doctrine  Pam  7,  2004:10). 
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Figure  1.1  System-of-Systems  Model  and  O-E-N-A-R  Chains 

Effects-Based  Planning  (EBP)  develops  the  integrated  application  of  selected 
instruments  of  power.  EBP  considers  all  domestic,  information,  military,  and  economic 
(DIME)  actions  available  to  commanders  in  order  to  influence  or  change  enemy  system 
behavior  or  capabilities  to  achieve  directed  policy  aims.  EBP  begins  when  the  planners 
clarify  commanders’  policy  aims  and  goals  and  translate  them  into  objectives.  Then, 
based  on  the  enemy  SoSA  model,  planners  determine  the  enemy  system  behaviors  and 
capabilities  that  need  to  be  influenced  or  changed.  The  desired  enemy  behaviors  and 
capabilities  are  described  as  a  set  of  effects  required  to  attain  the  objectives.  Essentially, 
an  effect  is  a  change  in  the  state  of  an  enemy  system.  Planners  look  to  change  the  state  of 
the  enemy  system  by  changing  the  state  of  enough  system  nodes.  Planners  target  the 
system  nodes  with  a  set  of  DIME  actions  to  produce  the  desired  effects  (i.e.,  desired 
change  in  system  behavior).  Then  planners  couple  available  resources  to  the  actions 
based  on  force  structure.  The  combination  of  the  objectives,  effects,  nodes,  actions,  and 
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resources  forms  O-E-N-A-R  chains.  These  O-E-N-A-R  chains  are  the  basis  of  Effects- 


Based  Courses  of  Action  (COAs)  available  to  commanders.  In  addition  to  the  desired 
effects,  planners  also  consider  the  secondary  and  unintended  effects  that  may  result  from 
changes  in  the  state  of  other  enemy  system  nodes.  Planners  can  then  change  the  COA  to 
accommodate  these  indirect  effects  (JWFC  Doctrine  Pam  7,  2004:10-12). 

Effects-Based  Execution  (EBE)  implements  the  actions  set  forth  in  EBP.  The 
subordinate  military  commanders  and  units  execute  the  military  actions  from  the  EBP, 
which  are  translated  as  tasks  via  orders  from  the  higher  commanders.  Other  supporting 
agencies  and  departments  are  tasked  via  the  interagency  process  to  perform  tasks  that 
accomplish  the  diplomatic,  infrastructure,  and  economic  actions  of  DIME  (JWFC 
Doctrine  Pam  7,  2004: 16). 

Effects-Based  Assessment  (EBA)  aims  to  identify  progress  towards  the 
attainment  of  commander’s  objectives  as  the  campaign  and  EBE  progresses.  EBA  forms 
the  basis  and  justification  for  changes  in  the  plan  and  future  execution.  EBA  tracks  both 
the  accomplishment  of  the  COA  actions  and  the  effects  of  those  actions  on  the  enemy.  In 
this  manner,  EBA  assesses  the  overall  campaign  helping  to  identify  trends  affecting 
future  operations.  EBA  assesses  the  overall  campaign  and  compares  the  current 
battlespace  picture  with  the  desired  battlespace  conditions  at  a  given  point  in  time.  When 
a  delta  exists  between  the  current  picture  and  desired  conditions,  further  analysis  is 
necessary  to  reveal  the  cause.  If  the  execution  has  followed  the  plan,  then  changes  to  the 
plan  maybe  required.  Conversely,  if  the  execution  has  not  followed  the  plan,  then  further 
analysis  must  reveal  the  cause,  and  the  commander  may  redirect  resources  to  remedy  the 
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situation.  The  results  of  EBA  then  feedback  into  EBP  (JWFC  Doctrine  Pam  7,  2004: 16- 
17). 

To  implement  the  whole  of  EBO,  the  process  must  include  a  SoSA  model,  EBP, 
EBE,  and  EBA.  The  bulk  of  the  EBO  tools  and  literature  created  thus  far  contribute  most 
to  the  SoSA  and  EBP  efforts.  However,  EBA  methodologies  remain  mostly  doctrinal. 
While  doctrinal  definitions  provide  a  framework  for  conceptualizing  EBA,  they  offer 
little  in  the  way  of  operational  implementation  with  regard  to  actually  assessing  the 
actions  and  effects  in  EBO. 

1.1.2  Effects-Based  Assessment  in  OPERA  TION  IRA  QI  FREEDOM. 

The  most  recent  large-scale  air  campaign  assessment  effort  occurred  in  2003 
during  the  first  days  of  OPERATION  IRAQI  FREEDOM  (OIF).  In  OIF,  planners 
attempted  to  implement  EBA  into  the  operational  assessment  process,  however,  the 
current  EBA  construct  was  not  conducive  to  the  rapid  pace,  short  duration,  highly 
dynamic  nature  of  OIF  (Allen,  2005  :v). 

At  the  height  of  air  operations,  the  Operational  Assessment  (OA)  team,  who  was 
responsible  for  conducting  EBA,  consisted  of  17  members.  Their  mission  was  to  track 
the  progress  of  the  CF  ACC’s  ten  Operational  Objectives  broken  into  over  200  Tactical 
Tasks  with  nearly  300  Measures  of  Effectiveness  (MOEs)  and  Success  Indicators  (Sis). 
MOEs  measured  the  progress  of  the  tactical  tasks  assigned  to  the  flying  units;  though 
called  MOEs,  the  metrics  actually  tracked  coalition  actions  and  not  effects.  Sis  measured 
progress  towards  operational  objectives  and  provided  broad  qualitative  guidance  for 
operational  assessment.  The  Sis  were  independent  of  the  MOEs  and  provided  evidence 
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of  effects  of  coalition  actions  on  the  enemy.  The  OAT  tracked  the  MOEs  and  Sis  daily, 
updating  the  metrics  iteratively  as  necessary.  Figure  1 .2  depicts  the  basic  strategy-to- 
task  hierarchy  used  for  the  Joint  Air  Operations  Plan  of  OIF  (Thoele,  DiSebastain,  and 
Garcia,  2004;  Allen,  2005:19). 


Figure  1.2  Strategy-to-Task  Hierarchy  used  in  OIF  (Thoele,  DiSebastian,  and  Garcia,  2004) 

This  process,  however,  failed  to  answer  the  questions  of  the  CFACC  and  did  not 
successfully  guide  and  steer  the  air  strategy  (Thoele,  2004a;  Allen,  2005:22).  In  his  2005 
School  of  Advanced  Air  and  Space  Studies  (SAASS)  thesis,  Major  Neil  Allen  pinpoints 
the  reasons  for  the  shortcomings  in  the  OA  team  process:  “In  a  short,  fast-paced  war  like 
OIF  .  .  .  the  EBA  construct  is  difficult  if  not  impossible  to  achieve  given  the  doctrine, 
organizational  structure,  and  technology  within  the  CAOC”  (Allen,  2005:3). 

The  organization  problems  that  hampered  the  EBA  efforts,  as  Allen  describes  them,  are 
stove-piped,  hierarchical  structures  and  insufficient  manning  levels  inside  the  CAOC, 
along  with  conflicting  interests  and  prioritizations  outside  the  CAOC.  Allen  also  points 
out  that  the  EBA  doctrine  was  lacking,  insufficient,  or  not  conducive  in  guiding  and 
instructing  assessors  in  the  EBA  process.  While  the  doctrine  described  what  to  do,  it  fell 


6 


short  of  explaining  how  to  do  it.  Without  a  defined  EBA  process  to  guide  them,  OA  team 
assessors  were  poorly  equipped  to  accomplish  EBA  during  OIF  (Allen,  2005:24,  38,  41). 

Therein  lies  the  biggest  problem  facing  the  analytical  community  with  regards  to 
EBA:  no  proven  assessment  methodology  exists  that  provides  insight  to  the  commander 
and  helps  guide  the  air  strategy.  Valid  complaints  of  the  current  EBA  methodology 
include  that  it  is  too  complex,  too  time  consuming,  and  too  dependent  on  intangible 
assessment  inputs  to  adapt  to  a  war  like  OIF.  In  order  to  guide  the  commander’s  strategy, 
assessments  need  to  be  timely,  accurate,  and  actionable.  Any  new  EBA  constructs  must 
be  operationally  useful  and  efficient  in  a  war  as  fast,  short,  and  dynamic  as  OIF.  Put 
another  way,  an  EBA  construct  must  be  useful  to  the  CFACC,  and  the  benefits  gained 
from  accomplishing  EBA  remain  greater  than  or  equal  to  the  overall  expenditure  of 
effort,  resources,  or  human  life.  The  challenge  then  is  to  fonnulate  an  EBA  construct  that 
is  timely,  accurate,  actionable,  efficient,  simple,  quantifiable,  adaptable,  and  most 
importantly,  provides  relevant  insight  to  the  CFACC  and  military  planners  (Allen, 

2005:9,  75-78). 

1.2  Problem  Statement 

To  provide  relevant  insight  to  the  CFACC  and  Strategy  Division  in  a  Combined 
Air  Operations  Center  (CAOC)  environment,  the  assessment  of  an  EBO  plan  and 
execution  must  include  an  assessment  of  the  effects  of  the  air  campaign  on  the  enemy  in 
addition  to  an  assessment  of  the  accomplishment  of  friendly  actions  taken  to  achieve  the 
desired  effects.  Detennining  the  effects  of  the  air  campaign  requires  an  analysis  of  the 
dynamics  of  the  enemy  systems.  EBA  must  be  able  to  recognize  the  state  of  the  enemy 
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systems  as  the  system  states  change  over  time.  This  thesis  advances  the  application  of 
EBA  by  defining  anticipated  states  of  enemy  systems,  developing  indicators  to  determine 
the  state  of  enemy  systems,  and  applying  progress  functions  to  the  anticipated  and  desired 
states  of  the  enemy  systems  in  order  to  describe  progress  of  the  campaign  towards 
attainment  of  the  commander’s  objectives. 

1.3  Research  Objective 

The  objective  of  this  thesis  is  to  establish  a  an  EBA  construct  that  supports  the 
EBO  process  by  providing  timely,  actionable  information  to  the  CFACC  and  CAOC 
staff.  The  construct  will  be  flexible  to  changes  in  the  operational  environment,  complex 
enough  to  provide  accurate  infonnation,  simple  enough  to  be  performed  on  existing 
CAOC  tools,  and  efficient  in  the  use  of  CAOC  resources.  In  addition,  this  thesis  defines 
the  necessary  phases  of  an  EBA  process  and  the  methods  to  be  applied,  setting  the  stage 
for  future  research  into  more  robust  stochastic  methods  and  the  development  of  future 
tools. 

1.4  Research  Scope 

This  thesis  focuses  on  the  assessment  of  an  air  campaign  at  the  operational  level 
of  warfare.  The  methodology  is  described  from  the  viewpoint  of  an  OA  team  within  a 
CAOC.  This  methodology  assesses  three  elements  of  EBO:  the  accomplishment  of 
CFACC  actions,  the  achievement  of  CFACC  desired  effects,  and  the  progress  towards  the 
attainment  of  CFACC  operational  objectives.  The  methodology  implicitly  considers  the 
constraints  of  time  and  personnel  on  an  OA  team,  and  therefore  is  designed  to  be  applied 
to  one  or  two  high-priority  commander  objectives.  The  methodology  explicitly  describes 


a  process  for  assessing  combat  operations,  though  the  methodology  is  intended  to  be 
general  enough  to  enable  its  application  to  stability  operations  as  well. 

1.5  Assumptions 

The  methodology  developed  in  this  thesis  makes  the  following  key  assumptions 
about  pre-conflict  analysis  of  the  enemy  and  data  availability  during  conflict: 

■  A  SoSA  model  exists  for  the  enemy  in  some  form,  and  this  model  is  used  by 
the  planners  to  develop  the  courses  of  action  examined  in  EBP.  Though  SoSA 
models  are  discussed  in  Chapter  2,  this  thesis  does  not  include  development  of 
a  SoSA  in  the  methodology. 

■  All  data  necessary  to  perform  the  assessments  described  in  this  methodology 
are  available  to  the  assessors.  Many  of  the  problems  described  by  Allen  and 
Thoele  et  al  deal  specifically  with  data  collection,  compilation,  and 
dissemination  within  the  CAOC.  This  thesis  views  these  issues  as 
organizational  constraints,  which  require  procedural  solutions,  and  are 
therefore  outside  of  the  scope  of  this  thesis.  For  further  analysis  and  research 
on  procedural  solutions  to  data  availability  for  CAOC  operations,  see  Allen 
(2005)  and  Air  Combat  Command  (2004). 

Other  assumptions  made  in  this  thesis  will  be  described  in  the  appropriate  position  within 
the  methodology. 

1.6  Overview  and  Format 

This  thesis  is  divided  into  five  chapters:  Chapter  1,  Introduction;  Chapter  2, 
Literature  Review;  Chapter  3,  Methodology;  Chapter  4,  Results  and  Analysis;  and 
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Chapter  5,  Discussion.  Chapter  2  presents  an  overview  of  EBO  and  discusses  in  detail  a 
formal  definition  of  effects,  including  different  types  and  levels  of  effects  as  they  apply  to 
warfare.  Chapter  2  also  presents  some  historical  examples  of  the  application  of  EBA  and 
then  reviews  relevant  methods  from  the  fields  of  medicine,  risk  analysis,  quality  control, 
and  decision  theory. 

Chapter  3  describes  the  mathematical  construct  of  EBA  and  presents  a  seven 
phase  methodology  for  performing  EBA.  Chapter  4  applies  the  EBA  methodology  to  two 
examples.  The  first  example  is  a  combat  operations  scenario  example  based  on  conflicts 
the  United  States  Air  Force  has  been  involved  in  over  the  last  two  decades.  The  second 
example  is  a  stability  operations  scenario  based  on  the  operations  involved  in  nation 
building  such  as  with  those  associated  with  OIF.  Chapter  5  then  discusses  the  limitations 
and  contributions  of  the  methodology,  as  well  as  areas  for  future  research. 
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2  Literature  Review 


2.1  Introduction 

Chapter  2  reviews  existing  research  and  publications  relevant  to  the  analysis 
methods  described  in  the  methodology  of  Chapter  3.  Chapter  2  first  looks  at  the 
definition  and  components  of  Effects-Based  Operations  (EBO)  and  a  formal  definition  of 
effects.  Next,  the  chapter  investigates  the  major  components  of  EBO,  namely 
Operational  Net  Assessment  (which  is  the  fonnal  Joint  process  that  performs  System-of- 
Systems  Analysis),  Effects-Based  Planning  (EBP),  Effects-Based  Execution  (EBE),  and 
Effects-Based  Assessments  (EBA).  Then  Chapter  2  reviews  the  contributions  of 
Operation  IRAQI  FREEDOM,  the  most  recent  large-scale  assessments  effort  of  an  air 
campaign.  Chapter  2  next  discusses  a  new  model  for  EBA  based  on  the  Jones  Criteria  for 
diagnosing  rheumatic  fever.  The  Jones  Criteria  model  leads  to  a  discussion  of  measures, 
indicators,  and  criteria  for  good  indicators  of  system  capability  and  behavior.  Next, 
Chapter  2  presents  tools  for  the  development  and  use  of  indicators  that  includes  cause- 
and-effect  diagrams  (fishbone  charts),  operational  experience,  and  Powell’s  40-70 
principle  for  decision  making  with  incomplete  information.  The  chapter  concludes  with  a 
discussion  of  the  contributions  of  risk  analysis  and  decision  theory,  and  the  subtle 
differences  between  value  functions  and  the  progress  functions  introduced  in  Chapter  3. 

2.2  EBO  Defined 

Several  definitions  for  EBO  exist.  Virtually  every  document  written  on  EBO 
contains  a  definition  with  a  flavor  unique  to  the  publication,  however,  each  definition  of 
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EBO  contains  the  same  general  concepts.  This  thesis  draws  primarily  from  the  United 
States  Joint  Forces  Command  Joint  Warfighting  Center  Joint  Doctrine  Series  Pamphlet  7; 
Mann,  Endersby,  and  Searle;  and  Smith  to  describe  EBO  and  its  components.  This  thesis 
begins  with  the  definition  of  EBO  prescribed  by  the  United  States  Joint  Forces  Warfare 
Center  Doctrine  Series  Pamphlet  7: 

Operations  that  are  planned,  executed,  assessed,  and  adapted  based  on  a 
holistic  understanding  of  the  operational  environment  in  order  to  influence 
or  change  system  behavior  or  capabilities  using  the  integrated  application 
of  selected  instruments  of  power  to  achieve  directed  policy  aims.  (JWFC 
Doctrine  Pam  7,  2004:2) 

This  thesis  considers  the  operational  environment  to  be  a  composite  of  all  elements, 
conditions,  and  influences  that  affect  the  employment  of  resources  and  capabilities  that 
bear  on  the  decisions  of  the  Combined  Forces  Air  Component  Commander  (CFACC).  In 
addition,  the  operational  environment  is  comprised  of  political,  military,  economic, 
social,  infrastructure,  and  information  (PMESII)  systems.  Analysis  of  these  systems  and 
their  interrelationships  provides  the  “holistic  understanding”  mentioned  in  the  definition 
(JWFC  Doctrine  Pam  7,  2004:2). 

A  system  is  defined  as  a  functionally,  physically,  and/or  behaviorally  related 
group  of  elements  that  interact  together  as  a  whole  (JWFC  Doctrine  Pam  7,  2004:2). 

This  thesis  will  refer  to  enemy  systems  that  will  change  behavior  or  capabilities.  When 
the  systems  being  described  belong  to  the  United  States  or  coalition  allies,  they  will  be 
denoted  as  friendly  systems. 

Instruments  of  power  can  include  diplomatic,  informational,  military,  economic 
(DIME),  and  other  means  available  to  national  leaders  to  influence  the  operational 
environment.  This  thesis  is  concerned  only  with  instruments  within  the  primary  control 
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of  the  CFACC,  that  is  military  and  infonnational  means.  Likewise,  directed  policy  aims 
refer  to  the  objectives  of  the  CFACC  and  the  Combined  Forces  Commander  (CFC)  that 
comprise  the  desired  operational  end  state  relevant  to  the  operation  at  hand  (JWFC 
Doctrine  Pam  7,  2004:3). 

In  summary,  for  the  purpose  of  this  thesis,  EBO  is  the  set  of  operations  planned, 
executed,  assessed,  and  adapted  based  on  a  holistic  understanding  of  the  operational 
environment  in  order  to  influence  or  change  enemy  system  behavior  or  capabilities  using 
the  integrated  application  of  selected  instruments  of  power  (military  and  informational)  to 
achieve  CFACC  and  CFC  objectives. 

2.3  Effects 

The  above  definition  of  EBO  speaks  of  influencing  or  changing  enemy  systems. 
These  changes  to  enemy  systems  embody  the  essence  of  effects.  Mann  et  al  describe  an 
effect  as  the  “full  range  of  outcomes,  events,  or  consequences  that  result  from  a  specific 
action”  (Mann,  Endersby,  and  Searle,  2002:31).  Similarly,  JWFC  Pamphlet  7  describes 
an  effect  as  “the  physical  and/or  behavioral  state  of  a  system  that  results  from  a  military 
and  nonmilitary  action  or  set  of  actions”  (JWFC  Doctrine  Pam  7,  2004:2).  The  key 
elements  of  an  effect  for  this  thesis  are  as  follows: 

1 .  A  system  of  interest  exists  in  an  initial  state. 

2.  The  system  transitions  to  a  different  state  as  a  result  of  an  action.  (To  be 
clear,  in  this  thesis,  “do  nothing”  can  be  an  action.) 

3.  The  resulting  change  in  the  state  of  the  system  is  the  effect. 
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2.3.1  First,  Second,  and  Third  Order  Effects. 


Effects  can  be  broken  down  into  direct  and  indirect  effects.  Direct  effects  are 
immediate  and  easily  recognizable.  Indirect  effects  are  often  the  cumulative  result  of 
many  direct  effects.  First-order  effects  are  direct  effects,  which  result  immediately  from 
an  action.  In  the  case  of  kinetic  operations,  results  of  the  action  are  directly  attributable 
to  military  attack  on  a  target  (Mann,  Endersby,  and  Searle,  2002:31-32). 

Second  and  third-order  effects  are  indirect  effects.  These  effects  result  from  one 
or  two  intermediate  effects  or  mechanisms,  thereby  producing  a  final  outcome.  A 
second-order  effect  has  one  intermediate  effect,  the  first-order  effect;  while  a  third-order 
effect  has  two  intermediate  effects,  the  first  and  second-order  effects.  Second  and  third- 
order  effects  tend  to  be  delayed  and  are  typically  more  difficult  to  recognize  than  direct 
effects  (Mann,  Endersby,  and  Searle,  2002:32-33). 

2.3.2  Cumulative  and  Cascading  Effects. 

Cumulative  effects  result  from  the  aggregate  of  many  direct  or  indirect  effects. 
Cumulative  effects  generally  occur  at  the  operational  and  strategic  levels  of  warfare, 
although  they  may  occur  at  the  tactical  level.  Cumulative  effects  can  be  considered  the 
“roll-up”  of  multiple  first,  second,  and  third-order  effects.  Cascading  effects  occur  when 
indirect  effects  flow  from  higher  levels  to  lower  levels  of  employment  (Mann,  Endersby, 
and  Searle,  2002:34).  Cascading  effects  can  be  considered  the  distribution  of  second  and 
third-order  effects  created  by  a  direct  or  first-order  effect  on  an  a  large  highly  connected 
system. 
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2.3.3  Collateral  Effects. 


An  effect  is  the  resulting  state  of  a  system  due  to  an  action.  Depending  on  the 
nature  of  the  system  state  and  whether  the  system  is  enemy  or  friendly,  the  effect  can  be 
positive  or  negative.  Collateral  effects  are  defined  as  any  outcome  (or  resulting  state  of  a 
system)  other  than  what  was  intended,  whether  positive  or  negative.  In  EBO,  collateral 
effects  should  be  a  major,  deliberate  consideration  in  planning,  executing,  and  assessing 
military  actions  (Mann,  Endersby,  and  Searle,  2002:35-36). 

2.3.4  Types  of  Effects  on  a  System. 

The  nature  of  the  system  and  the  action  applied  to  the  system  will  dictate  the 
resulting  state  of  the  system.  How  the  system  is  changed  by  the  action  can  be  described 
by  four  different  types  of  effects  on  a  system:  physical  effects,  functional  effects, 
systemic  effects,  and  psychological  effects. 

Physical  effects  are  direct,  first-order  effects.  Their  primary  purpose  is  to 
“damage,  disrupt,  or  neutralize  a  target  or  group  of  targets  through  the  application  of 
military  force”  (Mann,  Endersby,  and  Searle,  2002:37).  Physical  effects  result  in  a 
physical  alteration  of  the  system.  Functional  Effects  are  direct  or  indirect  effects  of 
actions  on  the  ability  of  a  target  or  system  to  function  properly  and  perform  its  mission. 
Systemic  effects  are  effects  that  are  aimed  at  disrupting  the  operation  of  a  specific  system 
or  set  of  systems.  Psychological  effects  are  the  results  of  actions  that  influence  emotions, 
motives,  objective  reasoning,  and  ultimately  the  behavior  of  individuals,  groups, 
organizations,  and  foreign  governments  (Mann,  Endersby,  and  Searle,  2002:37-38). 
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2.3.5  Effects  at  Different  Levels  of  Warfare. 


Changes  in  a  system  behavior  or  capability  can  occur  at  any  of  the  strategic, 
operational,  and  tactical  levels  of  warfare.  Both  the  nature  of  the  system  and  the  action 
applied  to  it  can  determine  the  level  of  the  effect.  Strategic  effects  contribute  to  affecting 
an  enemy’s  overall  political,  military,  and  economic  capacities  as  well  as  its 
psychological  stability.  Strategic  actions  are  activities  associated  with  the  campaign 
effort  as  a  whole.  Operational  level  actions  are  activities  that  affect  an  entire  theater  of 
operations.  The  focus  of  operational  actions  is  on  the  war-making  potential  of  the  enemy 
within  the  theater  of  operations.  Tactical  effects  are  the  result  of  an  action  or  actions  at 
the  individual  unit  level.  Tactical  effects  generally  occur  on  a  localized  basis  and  are 
immediate  and  short  in  duration  (Mann,  Endersby,  and  Searle,  2002:40-41). 

The  focus  of  this  thesis  is  detennining  the  impact  of  military  operations  at  the 
operational  level.  Generally  speaking,  operational  effects  can  be  direct  effects  and 
indirect  effects  of  tactical  effects.  Operational  effects  can  also  result  from  cumulative 
effects  from  tactical  actions  and  other  operational  effects  as  well  as  cascading  effects 
from  strategic  actions. 

2.3.6  Precisely  Defining  Effects. 

Gallagher  et  al  describe  the  necessity  of  precisely  defined  effects  for  the  proper 
execution  of  EBO.  Gallagher  et  al  state  that  analysts  must  be  able  to  identify  elemental 
effects  that  lend  themselves  to  analytical  modeling  and  analysis  in  order  for  analytical 
methods  to  be  applied  to  EBO.  Furthermore,  a  bottom-up  approach  for  EBO  planning 
and  assessment  requires  careful  definition  of  these  elemental  effects  such  that  they 
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achieve  exhaustive  coverage,  mutual  exclusivity,  and  independence  from  the  means  of 
achievement  of  the  effects  (Gallagher,  True,  and  Whiteman,  2004:9). 

A  problem,  however,  is  that  the  current  EBO  taxonomy  focuses  on  a  set  of  verbs  to 
describe  effects  (a  consequence  of  a  particular  action).  Few  of  the  proposed  EBO  verb 
sets  provide  detailed  definitions.  In  fact,  most  effects  implied  by  the  verb  sets  (terms)  are 
less  than  precise,  are  not  mutually  exclusive,  and  often  imply  kinetic  means  to  achieve 
the  effect.  That  is,  the  use  of  these  tenns  to  describe  effects  lacks  the  rigor  and  precision 
required  for  mathematical  analysis  (Gallagher,  True,  and  Whiteman,  2004:9).  Therefore, 
Gallagher  et  al  state  that  effects  should  not  be  defined  in  terms  of  verbs,  but  rather  should 
be  defined  as  an  impact  on  a  single  functional  capability  or  behavior  with  four 
specifications  (Gallagher,  True,  and  Whiteman,  2004:9): 

•  Range  (  a  capability-range  is  the  affected  area,  such  as  target,  city,  region  or  state, 
whereas  behavior-range  is  the  affected  individual,  group  or  nation); 

•  Extent  (specifies  the  resulting  level  of  capability  or  behavior  whether  it  is 
decreased,  maintained  or  increased); 

•  Start  time; 

•  End  time. 

This  thesis  uses  the  specifications  for  precisely  defining  effects  outlined  by 
Gallagher  et  al  as  the  initial  step  in  a  comprehensive  EBA  methodology.  Chapter  3 
describes  how  well-defined  desired  effects  are  used  to  direct  the  EBA  process.  With 
regards  to  the  analytical  requirements  of  the  effects  definition  (exhaustive  coverage, 
mutually  exclusivity,  and  independence  from  the  means  of  achievement),  this  thesis 
discusses  neither  the  need  for  the  effects  to  be  independent  from  the  means  of 
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achievement  nor  the  need  for  mutual  exclusivity.  While  exhaustive  coverage  directly 
supports  the  EBA  method  through  the  assessment  of  the  higher  objectives,  independence 
from  the  means  of  achievement  primarily  supports  the  planning  process  to  determine 
actions.  Likewise,  mutually  exclusivity  of  effects  is  an  invalid  assumption  when  effects 
are  viewed  through  the  interconnected  system-of-systems  model.  Achieving  one  effect 
may  aid  or  deter  the  achievement  of  other  effects,  and  therefore  effects  are  not  mutually 
exclusive. 

Desired  effects  may  be  defined  in  terms  of  two  different  sets  of  consequences: 
those  to  be  achieved  and  those  to  be  avoided  (Gallagher,  True,  and  Whiteman,  2004:15). 
JWFC  Pamphlet  7  states  that  a  “specified  effect  describes  desired  or  undesired  conditions 
.  .  .  that  results  fonn  a  set  of  actions  (JWFC  Doctrine  Pam  7,  2004:12).”  Therefore, 
planners  must  consider  both  types  of  effects  and  the  systems  they  involve  when 
decomposing  the  commander’s  objectives  into  a  set  of  effects.  This  thesis  considers  the 
contributions  of  undesired  and  collateral  effects  in  the  overall  assessment  methodology. 

2.4  Components  of  EBO 

An  EBO  methodology  is  a  means  for  planning,  executing,  and  assessing 
operations  designed  to  attain  the  effects  required  to  achieve  desired  national  security 
outcomes  (Mann,  Endersby,  and  Searle,  2002:46).  An  EBO  methodology  relies  heavily 
on  a  comprehensive  system-of-systems  understanding  of  the  operational  environment. 
Therefore,  the  EBO  process  begins  with  a  complete  system-of-systems  understanding  of 
the  enemy  and  proceeds  with  three  highly  interdependent  and  overlapping  major 
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components:  effects-based  planning,  effects-based  execution,  and  effects-based 
assessment  (JWFC  Doctrine  Pam  7,  2004:8-9). 

2.4. 1  Operational  Net  Assessment  and  Systeni-of-Systems  Analysis. 

Virtually  no  part  of  a  target  system  or  its  infrastructure  is  isolated,  therefore  it  is 
important  to  develop  a  better  understanding  of  inherent  relationships  of  effects  to 
incorporate  understanding  into  planning,  execution,  and  assessment  of  EBO  (Mann, 
Endersby,  and  Searle,  2002:34,  39).  In  order  to  build  a  holistic  knowledge  base  of  the 
operational  environment,  EBO  begins  with  a  System-of-Systems  Analysis  (SoSA)  of  the 
enemy  PMESII  systems.  Operational  Net  Assessment  (ONA)  is  the  organizational 
process  that  performs  the  SoSA  and  develops  the  SoSA  model  (JWFC  Doctrine  Pam  7, 
2004:9). 

United  States  Joint  Forces  Command  defines  ONA  as  “the  integration  of  people, 
processes,  and  tools  that  use  multiple  information  sources  and  collaborative  analysis  to 
enhance  command  decision-making.”  The  aim  of  ONA  is  to  produce  a  coherent,  relevant 
and  shared  knowledge  environment  for  planners  and  decision  makers.  In  doing  so,  ONA 
uses  link  analysis,  network  analysis  and  structured  augmentation  to  assess  an  adversary’s 
PMESII  systems,  thus  producing  a  SoSA  model.  The  SoSA  model  reveals  critical  nodes 
and  vulnerabilities  that  may  be  used  in  EBO.  In  order  to  create  the  SoSA  model,  ONA 
integrates  military,  national  agencies,  coalition  governments,  nongovernmental 
organizations  and  other  partners  who  have  information  to  contribute.  ONA  is  a 
continuous  and  dynamic  process  that  operates  through  peacetime  and  conflict.  ONA 
enables  commanders  to  avoid  conflict  by  engaging  opponents  in  influence  and  deter 
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methods;  likewise,  ONA  offers  defeat  mechanisms  to  accomplish  commanders’ 
objectives  during  times  of  conflict.  In  this  manner,  ONA  provides  the  first  crucial  piece 
of  EBO,  the  SoSA  model  of  enemy  systems  (USJFCOM,  2006). 

A  SoSA  model  provides  a  framework  for  understanding  the  battlespace  by 
characterizing  the  complex  organization,  relationships,  and  key  characteristics  of  an 
enemy  PMESII  network.  SoSA  recognizes  the  linkages  across  the  various  areas  of  an 
enemy  system  and  describes  the  connections  and  key  components  of  the  entire  system.  A 
SoSA  model  decomposes  the  system  of  interest  into  a  network  of  nodes  and  links  (Figure 
2.1).  A  node  can  be  a  person,  place,  or  any  physical  thing  that  is  a  fundamental 
component  of  the  system.  Links  represent  the  behavioral,  physical,  or  functional 
relationship  between  the  nodes.  These  links,  are  the  basis  for  the  causal  linkages,  which 
describe  why  planners  think  the  proposed  actions  will  achieve  the  desired  effects.  A 
SoSA  model  increases  the  understanding  of  how  actions  taken  against  one  element  of  the 
system  can  affect  the  entire  system  as  well  as  other  elements  of  the  system.  (JWFC 
Doctrine  Pam  7,  2004:10;  Mann,  Endersby,  and  Searle,  2002:49) 


PMESII  System  ,nformation 


Figure  2.1  System  of  Systems  Model  of  a  PMESII  System 
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2.4.2  Example  SoSA  Models. 


Building  SoSA  models  brings  additional  analytical  challenges  to  the  operational 
implementation  of  EBO.  Several  authors  have  offered  techniques  and  tools  to  facilitate 
the  building  of  a  SoSA  model.  This  thesis  examines  briefly  two  SoSA  methods  for 
building  models,  one  by  Lee  and  Kupersmith  and  the  other  by  Goodwinan  and  Lee.  Lee 
and  Kupersmith  describe  a  hierarchical  structure  for  a  SoSA  model  based  on  their 
Objectives  to  Metrics  Methodology  (OMM);  Goodwin  and  Lee  describe  a  more  robust 
version  of  a  SoSA  model,  the  Net-Centric  Effects-Based  Operations  Model  (NEMO), 
which  is  based  on  existing  engineering  models  of  infrastructure  networks.  Together  these 
methods  represent  the  broad  spectrum  of  complexity  of  SoSA  models,  with  OMM  being 
a  simpler  form  of  a  SoSA  model  and  NEMO  being  a  more  complex  fonn.  In  this  thesis 
models  that  represent  a  system  on  the  basis  of  topology  alone  are  considered  to  be 
simple;  models  that  require  engineering-level  detail  to  represent  the  system  are 
considered  to  be  complex. 

Lee  and  Kupersmith  base  their  SoSA  frameworks  on  two  broad  conditions  under 
which  desired  effects  can  be  achieved:  1)  the  enemy  “can’t”  achieve  their  own  desired 
objectives;  2)  the  enemy  “won’t”  perform  activities  in  support  of  their  objectives. 

“Can’t”  speaks  to  the  enemy’s  capability.  “Won’t”  speaks  to  the  enemy’s  will.  These 
conditions  are  a  crucial  part  of  the  Objectives  to  Metrics  Methodology  (OMM).  OMM  is 
a  five  step  process  designed  to  link  a  desired  effect  to  a  measurable  set  of  metrics  (Lee 
and  Kupersmith,  2002:4-5). 

In  OMM,  planners  first  identify  the  objective  based  on  the  Commander’s  Intent 
Statement  for  a  given  operation,  which  is  generally  used  to  describe  the  end-state  of  an 
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operation.  Second,  planners  define  the  effect  using  doctrinal  definitions,  because  these 
definitions  provide  a  fonnal  common  frame  of  reference.  Third,  planners  frame  the 
effect  in  quantifiable  tenns  that  help  detennine  if  the  effects  are  being  accomplished  or 
not.  To  do  this,  the  intent  of  an  effect  is  reformed  into  a  question  to  facilitate  the 
establishment  of  metrics  against  an  objective.  For  example,  for  the  objective  Air 
Superiority,  the  desired  effect  is  freedom  of  movement  in  area  of  interest.  The  question 
then  would  be  phrased  as  “Can  BLUE  conduct  air  ops  without  interference  from  RED?” 
Fourth,  planners  assign  target  sets  to  the  effect,  decomposing  each  of  the  can ’t  and  won  7 
ideas  into  details  from  which  vulnerabilities  and  interactions  can  be  identified  and 
prioritized  for  attack.  It  is  here  that  Lee  and  Kupersmith  utilize  their  framework  structure 
of  a  SoSA  model  (Lee  and  Kupersmith,  2002:4-8). 

The  Lee  and  Kupersmith  model  (Figure  2.2)  consists  of  elements  comprising  each 
effect.  This  structure  helps  to  visualize  the  interrelationships  of  the  various  factors  that 
contribute  to  an  effect  while  simultaneously  accounting  for  key  factors.  In  addition,  the 
model  helps  the  planner  determine  potential  vulnerabilities  for  attaining  a  desired  effect 
with  acceptable  consumption  of  resources.  This  model  looks  at  effects  from  both  the 
can  7  and  won  7  conditions  to  gain  insight  into  key  linkages  between  the  sides  of  the 
framework.  It  is  important  that  the  resulting  tree  structure  be  sufficiently  detailed  to 
define  targets  and  actions  (kinetic  or  non-kinetic)  associated  with  neutralizing  them  (Lee 
and  Kupersmith,  2002:10). 
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Figure  2.2  Enemy  Systems  Framework  (Lee  and  Kupersmith,  2002:10) 

The  fifth  and  final  step  of  OMM  is  to  establish  metrics  to  measure  achievement  of 
the  effect.  The  metrics  should  be  considered  synergistically  in  order  to  measure  progress 
in  achieving  an  effect  (Lee  and  Kupersmith,  2002:8).  This  thesis  views  the  culmination 
of  effects  to  be  a  non-linear  function,  and  agrees  with  Lee  and  Kupersmith  that  the  effects 
should  be  viewed  synergistically. 

Goodwin  and  Lee  describe  an  automated  approach  for  performing  SoSA  that 
leverages  existing,  commercially  available  software  models — the  Net-Centric  Effects- 
Based  Operations  Model  (NEMO).  NEMO  provides  analysts  with  the  means  for  defining 
relationships  between  multiple  infrastructure  networks  within  a  single  user  interface. 
NEMO  has  the  capability  to  model  interactions  across  electrical  power,  water,  gas,  and 
road  networks  using  “on/off  interaction  behavior”  between  the  components  of  the 
different  networks.  In  this  way  NEMO  provides  a  basic  capability  for  performing 
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effects-based  planning  and  analysis  for  operations  against  an  opponent’s  physical 
infrastructure  network  (Goodwin  and  Lee,  2005:4-5,  13,  15). 

NEMO  enables  planners  to  consider  a  target  as  a  component  of  the  network 
(regardless  of  the  target  type)  by  providing  a  framework  to  model  an  opponent’s  PMESII 
network  as  well  as  their  interdependencies.  Specifically,  NEMO  integrates  infrastructure 
models  such  as  lines  of  communication,  electrical  power,  gas  pipelines,  and  water 
pipelines.  NEMO  can  help  identify  nodal  intersections  between  system  layers  using  a 
geospatial  database;  intersections  are  where  different  infrastructure  layers  lie  within  a 
user-specified  distance  from  each  other.  In  this  way,  planners  can  understand  how  a 
particular  element  of  one  network  relies  on  the  elements  of  one  or  more  other  networks 
thus  providing  a  basis  for  analyzing  cascading  effects.  Understanding  and  quantifying 
the  nature  of  these  relationships  is  the  key  to  performing  effects-based  analysis  (Goodwin 
and  Lee,  2005:5,  7-8). 

An  example  of  NEMO  use  is  the  analyzing  the  effect  of  bringing  down  electrical 
power  on  the  travel  time  between  points  A  and  B  in  Figure  2.3.  The  dashed  line 
represents  the  shortest  path.  Taking  the  power  down  at  point  D  affects  the  draw  bridge  at 
point  C  on  the  graph.  As  a  result,  the  shortest  path  has  changed  as  represented  by  the 
dashed  yellow  line  in  Figure  2.4. 
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Figure  2.3.  NEMO  Analysis  of  Electrical  Outage  Impact  on  Travel  Time  (Goodwin  and  Lee, 

2005:11) 


One  drawback  of  NEMO  is  that  it  uses  highly  detailed  physical  engineering  models, 
which  require  users  with  a  significant  amount  of  domain  experience  and  knowledge  of 
the  infrastructure  being  modeled.  Knowing  only  the  general  network  topology  of  a 
network  is  not  sufficient  to  support  the  highly  detailed  modeling  of  NEMO.  However, 
for  use  in  EBO  a  SoSA  model  may  not  require  engineering  level  detail  (Goodwin  and 
Lee,  2005:12). 

The  search  for  other  methods  of  creating  SoSA  models  includes  those  methods  as 
complex  and  detail  specific  as  NEMO  and  those  that  are  more  simplified  as  with  OMM. 
Umstead  offers  a  literature  review  of  operations  research  methods  to  aid  the  analysis  and 
building  of  SoSA  models.  These  methods  include  Bayesian  networks,  influence 
diagrams  and  networks,  social  networks,  game  theory,  complexity  theory  and  others 
(Umstead,  2005,  56-69). 
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Figure  2.4.  Resulting  Time  Delays  in  New  Route  Due  to  Power  Outage  (Goodwin  and  Lee, 

2005:12) 


These  examples  of  SoSA  models  and  analytical  methods  offer  a  wide  variety  of 
starting  points  for  developing  a  descriptive  model  of  the  adversary  and  an  improved 
understanding  of  the  relationships  and  linkages  between  components  of  the  adversary 
system  (Umstead,  2005:21).  However,  performing  SoSA  and  developing  a  SoSA  model 
is  outside  the  scope  of  this  thesis.  For  the  purposes  of  the  assessment  methodology 
presented  in  Chapter  3,  this  thesis  assumes  that  a  SoSA  model  exists  in  some  form  prior 
to  the  start  of  the  assessment  process.  In  Chapter  4,  the  examples  presented  are  based  on 
a  simpler  form  of  the  SoSA  models.  This  thesis  views  a  simpler  SoSA  model  as 
appropriate  at  the  operational  level  because  the  simpler  form  of  the  model  requires  less 
lead  time  to  build  and  still  maintains  the  fidelity  that  is  required  at  the  operational  level. 
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2.4.3  Effects-Based  Planning. 


The  goal  of  Effects-Based  Planning  (EBP)  is  to  produce  an  executable  course  of 
action  (COA).  The  COAs  decompose  into  five  critical  components:  objectives,  effects, 
nodes,  actions,  and  resources.  The  combination  of  the  objectives,  effects,  nodes,  actions, 
and  resources  forms  O-E-N-A-R  chains.  These  O-E-N-A-R  chains  from  the  basis  of  the 
COAs  available  to  commanders.  The  EPB  process  results  in  a  set  of  COAs  which 
includes  measures  for  detennining  success  as  well  as  the  traditional  task,  purpose,  and 
desired  end  state,  which  have  been  traditionally  used  in  operational  planning  (JWFC 
Doctrine  Pam  7,  2004:8,  10,  12). 

EBP  begins  when  strategists  and  planners  clarify  commanders’  goals  and 
objectives.  Objectives  define  an  end  state  that  actions  are  designed  to  achieve.  The 
objectives  are  the  first  crucial  piece  of  the  COA.  The  objectives  are  then  further  defined 
by  a  set  of  conditions  or  desired  effects  that  must  be  created  to  achieve  each  objective. 
These  desired  effects  require  foreknowledge  of  specific  achievable  conditions  believed 
necessary  for  attaining  the  objectives.  For  each  objective  in  EBP,  planners  detennine  the 
set  of  effects  that  are  required  to  attain  the  objective.  Here,  the  SoSA  model  helps 
determine  what  PMESII  behaviors  or  capabilities  (effects)  are  required  to  achieve 
objectives.  Effects  are  the  second  piece  of  the  O-E-N-A-R  chains.  (JWFC  Doctrine  Pam 
7,  2004:7,  10-12;  Mann,  Endersby,  and  Searle,  2002:  44) 

The  third  link  in  the  O-E-N-A-R  chain  is  a  node.  Using  the  SoSA  model, 
planners  determine  which  nodes  of  the  enemy  system  apply  to  the  effects.  Planners  also 
consider  the  secondary  and  unintended  effects  that  may  result  from  changes  in  the  state  of 
the  nodes  (JWFC  Doctrine  Pam  7,  2004: 10-12).  When  planners  carefully  consider  the 
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desired  effects  that  must  be  established  to  achieve  objectives  and  the  underlying  causal 
linkages  between  nodes,  they  may  find  that  potential  negative  collateral  effects  outweigh 
the  positive  intended  effects  (Mann,  Endersby,  and  Searle,  2002:44). 

For  each  node,  planners  can  apply  a  set  of  actions  to  achieve  the  desired  effects. 
EBP  considers  all  DIME  actions  available  to  commanders  in  order  to  create  desired 
effects  that  will  achieve  the  commanders’  objectives.  Planners  aim  to  harmonize  DIME 
actions  to  influence  PMESII  systems  in  order  to  achieve  desired  effects,  which  leads  to 
attainment  of  operational  objectives  (JWFC  Doctrine  Pam  7,  2004:8).  Analysis  of  the 
causal-linkages,  which  describe  why  an  action  will  achieve  an  effect,  can  help  in 
understanding  the  potential  contribution  of  a  particular  action  towards  attaining 
objectives  (Mann,  Endersby,  and  Searle,  2002:49).  Finally,  planners  couple  the  actions 
to  available  resources  and  forces  (JWFC  Doctrine  Pam  7,  2004:10-12),  thus  completing 
the  O-E-N-A-R  chain  of  a  COA.  This  thesis  does  not  discuss  the  constraint  of  resources. 
The  assumption  is  that  the  strategists  and  planners  have  all  necessary  resources  available 
to  them  in  order  to  execute  a  chosen  COA. 

2.4.4  Effects-Based  Execution. 

Effects-Based  Execution  (EBE)  implements  the  actions  set  forth  in  EBP.  The 
subordinate  military  commanders  and  units  execute  the  actions  from  the  EBP,  which  are 
translated  as  tasks  via  orders  from  the  higher  commanders.  In  the  case  of  an  air 
campaign,  the  CFACC  produces  the  Air  Tasking  Order  (ATO),  which  orders  the 
subordinate  units  to  execute  the  actions.  All  data  necessary  to  assess  a  combat  operations 
will  be  generated  by  EBE.  This  data  includes  but  is  not  limited  to  mission  reports 
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(MISREPS),  Battle  Damage  Assessments  (BDA),  order  battle  charts,  message  traffic, 
intelligence  summaries  (INTSUMs),  etc.  All  of  these  sources  of  data  will  aid  the 
assessment  of  the  execution  of  military  and  infonnation  actions  and  the  effects  those 
actions  create. 

2.4.5  Effects-Based  Assessment. 

Effects  Based  Assessment  (EBA)  aims  to  identify  progress  towards  attainment  of 
commander’s  objectives  as  the  campaign  and  EBE  progresses.  EBA  forms  the  basis  and 
justification  for  changes  in  the  plan  and  execution.  EBA  compares  the  current 
battlespace  picture  with  the  desired  battlespace  conditions  at  any  point  in  time.  When  a 
delta  exists  between  the  current  picture  and  desired  conditions,  further  analysis  reveals 
the  cause.  If  the  execution  has  followed  the  plan,  then  changes  to  the  plan  maybe 
required.  Conversely,  if  the  execution  has  not  followed  the  plan,  then  further  analysis 
must  reveal  the  cause,  and  the  commander  may  redirect  resources  to  remedy  the  situation 
(JWFC  Doctrine  Pam  7,  2004: 16-17).  Each  piece  of  the  EBA  process  will  contribute  to 
the  methodology  presented  in  Chapter  3  of  this  thesis. 

Several  types  of  assessment  exist  to  aid  EBA,  including  physical  assessment, 
target  system  assessment,  performance  assessment,  and  psychological  assessment. 
Physical  assessment  of  an  individual  target  looks  at  the  direct,  first-order  effect  of  a 
kinetic  military  action.  Physical  assessment  is  a  necessary  component  of  EBA  and 
focuses  on  specific,  enumerable  critical  nodes.  Physical  assessment  determines  which 
nodes  of  the  system  are  still  functioning  to  ensure  the  desired  effect  of  the  system  is 
achieved  (Smith,  2002:358).  Physical  assessments  are  then  used  to  perform  target  system 
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assessment;  the  physical  assessment  information  is  fused  with  functional  damage  to  a 
target  system  and  an  evaluation  is  made  of  the  overall  impact  on  the  system’s  capabilities 
(Mann,  Endersby,  and  Searle,  2002:38). 

Performance  assessment  involves  monitoring  the  physical  performance  of  a 
system  over  time  and  comparing  the  baseline  performance  to  the  perfonnance  post¬ 
action.  Performance  assessment  is  largely  quantifiable.  System  performance  metrics 
look  at  the  culmination  of  indirect  physical  effects  planners  hoped  would  grow  from 
destruction  of  the  key  nodes  in  the  system.  The  critical  element  in  perfonnance 
assessment  is  less  the  collection  of  new  data  on  the  system  than  it  is  the  availability  of  a 
data  or  knowledge  base  from  which  to  calculate  change  (Smith,  2002:358-359,  366). 

When  dealing  with  large  systems,  the  effects  will  cross  over  into  the 
psychological  domain,  which  is  considered  in  psychological  assessment.  Psychological 
assessment  operates  on  two  levels:  physical  indicators  (point  indicators)  and  indirect 
psychological  effects  determined  by  intuition  and  mental  models  of  decision  makers  and 
subject-matter  experts.  Psychological  effects  are  extremely  difficult  to  measure 
accurately,  although  performance  assessment  can  be  used  to  assess  behavioral  changes, 
which  could  indicate  psychological  effects  (Smith,  2002:367,  370-372;  Mann,  Endersby, 
and  Searle,  2002:39). 

The  EBA  methodology  of  Chapter  3  does  not  specifically  dictate  a  process  for 
performing  the  above  listed  assessments,  however  these  assessments  are  very  important 
to  the  completion  of  the  methodology.  To  perfonn  EBA,  each  of  these  assessments  plays 
a  vital  role  in  determining  the  status  of  the  enemy  systems  under  consideration  in  EBO. 
The  methodology  presented  in  this  thesis  views  the  actual  assessment  of  the  enemy 
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systems  as  a  function  of  the  Intelligence,  Surveillance,  and  Reconnaissance  Division 
within  the  CAOC.  The  methodology  then  uses  these  assessments  as  data  to  perform  an 
overall  effects  assessment. 

2.5  EBA  in  OPERATION  IRAQI  FREEDOM 

The  assessment  methodology  used  during  the  air  war  of  OPERATION  IRAQI 
FREEDOM  (OIF)  is  the  most  recent  example  of  a  large-scale  assessment  of  an  air 
campaign  at  the  operational  level.  The  methodology  used  in  OIF  offers  a  starting  point 
for  developing  future  methods  such  as  the  one  in  this  thesis.  The  methodology  used  in 
OIF  implemented  many  but  not  all  of  the  items  listed  previously  in  this  chapter.  The 
Joint  Air  Operations  Plan  (JAOP)  was  based  on  a  Strategy-to-Task  hierarchy  developed 
by  the  strategic  planners  with  input  from  assessors.  The  Strategy-to-Task  hierarchy 
consisted  of  Operational  Objectives,  Tactical  Objectives,  Tactical  Tasks,  Success 
Indicators  (SI),  Focus  of  Effort,  Desired  Effects,  and  MOEs  (Figure  2.5).  The  JAOP 
contained  ten  Operational  Objectives  that  were  decomposed  into  200+  Tactical  Tasks. 

The  assessors  planned  to  assess  the  entire  JAOP  on  a  daily  basis.  To  handle  the 
workload  associated  with  assessing  a  campaign  of  that  magnitude,  the  assessors  assigned 
Offices  of  Primary  Responsibility  (OPR)  and  Subject  Matter  Experts  (SME)  in  all 
sections  of  the  CAOC.  The  OPRs  and  SMEs  helped  to  develop  the  Tactical  Tasks  and 
MOEs  as  well  as  provide  the  data  needed  to  assess  their  accomplishment.  At  the  task 
level,  assessors  defined  quantitative  MOEs  to  determine  the  task  accomplishment  and  the 
achievement  of  first-order  desired  effects. 
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Figure  2.5  Example  Strategy-to-Task  Hierarchy  used  in  OIF  (Thoele,  DiSebastian,  and 

Garcia,  2004) 

At  the  objective  level,  assessors  defined  Sis  that  represented  evidence  of  effects 
and  the  attainment  of  objectives.  The  Sis  were  a  more  subjective  assessment  than  MOEs 
and  were  assessed  independently  of  the  MOEs.  The  assessors  along  with  planners, 

OPRs,  and  SMEs  developed  over  300  MOEs  and  Sis.  The  Focus  of  Effort  and  Desired 
Effects  (not  pictured)  were  planning  tools  used  to  guide  the  Air  Tasking  Order  cycle  and 
were  not  used  in  the  assessment  process  (Thoele,  DiSebastian,  and  Garcia,  2004). 

To  assess  the  overall  campaign,  the  assessors  and  planners  applied  weights  to 
each  level  of  the  hierarchy.  As  the  OPRs  provided  data,  the  assessors  calculated  the 
MOE  scores  along  with  the  hierarchy  weights  in  an  additive  value  function.  The  result 
was  a  normalized  rating  between  0  and  1 ;  0  being  the  lowest  rating  and  1  being  the 
highest.  This  mathematical  rating  along  with  a  subjective  assessment  was  then  translated 
into  a  stoplight-chart  assessment  for  the  Tactical  and  Operational  Objectives  (Figure  2.6). 
As  the  campaign  progressed,  however,  this  assessment  process  and  methodology  had  to 
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be  put  aside  in  order  to  solve  other  CAOC  issues  and  provide  the  CFACC  with  answers 
to  other  pressing  questions  (Thoele,  DiSebastian,  and  Garcia,  2004). 
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Figure  2.6  Example  Assessment  from  OIF  (Thoele,  DiSebastian,  and  Garcia,  2004) 

Assessing  the  JAOP  each  day  required  assessors  to  make  determinations  on  the 
CF ACC’s  top  100  prioritized  Tactical  Tasks.  This  equated  to  gathering  data  for  150  to 
200  MOEs  and  Sis  daily.  Even  with  SMEs  and  OPRs  throughout  the  CAOC,  and  a 
robust  in-house  database  created  by  the  OA  team  specifically  for  their  mission,  the  17- 
member  OA  team  could  not  keep  up  with  all  of  the  metrics.  The  number  of  metrics  to  be 
tracked  was  just  too  great  for  the  OA  team  to  track  comprehensively  on  a  daily  basis. 

The  number  of  metrics  (approximately  150)  provides  this  thesis  with  an  upper  bound  for 
the  number  of  metrics  and  indicators  capable  of  being  tracked  in  practice.  The 
methodology  presented  in  Chapter  3  states  the  tractable  number  of  indicators  to  be 
tracked  is  50  to  60,  based  on  the  recommendations  of  Thoele  et  al  (Thoele,  DiSebastian, 
and  Garcia,  2004). 


33 


Though  called  “Measures  of  Effectiveness,”  the  MOEs  actually  tracked  coalition 
efforts  and  not  effects.  Of  the  200  tasks  there  were  only  five  distinct  types  of  tasks,  each 
with  a  corresponding  type  of  MOE.  Table  2. 1  summarizes  the  tasks  and  MOEs  used  to 
assess  the  air  campaign.  This  thesis  uses  the  MOE  structure  from  OIF  to  aid  actions 
assessment  as  a  part  of  the  overall  EBA  methodology.  The  methodology  presented  in 
Chapter  3  designates  measures  of  performance  (MOP)  for  tracking  friendly  actions. 
These  MOPs  are  based  upon  the  fonnat  of  four  of  the  MOEs  outlined  in  the  OIF 
methodology  (Thoele,  DiSebastian,  and  Garcia,  2004). 


Table  2.1  Example  Tasks  and  MOEs  used  in  OIF  (Thoele,  DiSebastian,  and  Garcia,  2004) 


Task 

MOE 

Degrade/ disrupt/destroy/neutralize  the 
enemy. 

Percentage  of  the  enemy 

degraded/disrupted/ 

destroyed/neutralized. 

Influence  the  enemy. 

Enemy  behaves  as  desired. 

Locate,  monitor,  and  track  enemy  units. 

Percentage  of  known  enemy  units 
located/monitored/tracked. 

Detect  and  identify  specific  enemy 
capabilities  (GPS  jamming,  etc.) 

Number  of  occurrences  of  enemy 
capability  detected  and  identified. 

Maintain  24-hour  coverage/specific 
capability. 

Percentage  of  coverage/capability 
maintained. 

2.6  Jones  Criteria  as  a  Model  for  EBA 

This  thesis  now  takes  a  turn  towards  another  way  of  thinking  about  assessing 
effects,  borrowing  a  model  methodology  from  the  field  of  medicine.  In  1944,  Dr  T. 
Duckett  Jones  published  a  set  of  criteria  for  the  diagnosis  of  rheumatic  fever  (Table  2.2). 
The  modified  Jones  Criteria  fonn  the  basis  for  the  diagnosis  criteria  that  are  still  used 
today.  Rheumatic  fever  is  known  to  be  related  to  previous  infection  with  group  A  [>- 
hemolytic  streptococci,  but  how  the  disease  operates  in  the  patient,  or  its  mechanism,  is 
unknown.  No  specific  laboratory  diagnostic  test  exists,  and  distinguishing  the  disease 
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from  other  diseases  is  sometimes  impossible.  Therefore,  the  diagnosis  of  rheumatic  fever 
must  be  arbitrary  and  empirical  (Rutstein  et  al,  1956;  Rheumatic  Fever,  2005). 

Jones  proposed  a  list  of  criteria  divided  into  major  and  minor  categories. 
Categorization  of  the  criteria  depended  on  their  relative  occurrence  in  rheumatic  fever 
and  their  relative  occurrence  in  other  disease  syndromes,  which  need  to  be  differentiated 
from  rheumatic  fever.  For  example,  chorea  is  included  in  the  major  criteria,  while  fever, 
a  symptom  common  to  many  diseases,  is  included  in  the  minor  criteria  (Rutstein  et  al, 
1956). 


Table  2.2  Jones  Criteria  for  Diagnosis  of  Rheumatic  Fever  (Rutstein  et  al,  1956) 


Major  Criteria 

Minor  Criteria 

I.  Carditis 

I.  Fever 

II.  Polyarthritis 

II.  Arthalgia 

III.  Chorea 

III.  Prolonged  P-R  Interval  in  the 

Electrocardiogram 

IV.  Subcutaneous  Nodules 

IV.  Increased  Erythrocyte  Sedimentation  Rate, 
Presence  of  C-reactive  Protein  or  Leukocytosis 

V.  Erythema  Marginatum 

V.  Evidence  of  Preceding  Beta-hemolytic 
Streptococcal  Infection 

VI.  Previous  History  of  Rheumatic  Fever  or  the 
Presence  of  Inactive  Rheumatic  Heart  Disease 

The  use  of  the  criteria  dictates  that  “the  presence  of  two  major  criteria  or  one 
major  and  two  minor  criteria  indicates  a  high  probability  of  the  presence  of  rheumatic 
fever”  (Rutstein  et  al,  1956).  Additionally,  the  criteria  are  not  meant  to  substitute  the 
judgment,  wisdom,  and  experience  of  the  physician.  The  criteria  are  designed  only  to 
guide  the  diagnosis  of  the  disease  (Rutstein  et  al,  1956). 

The  diagnosis  of  rheumatic  fever  presents  challenges  similar  to  EBA.  Assessors 
look  to  define  the  current  state  of  enemy  and  coalition  systems  in  order  to  detennine 
progress  towards  desired  effects  and  objectives.  Often,  assessors  know  a  cause  or 
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condition  related  to  the  desired  end  state  but  have  no  direct  indicator  to  detennine  the 
current  state  of  a  system.  Also,  how  a  system  moves  from  state  to  state  is  often  unknown. 
Likewise,  distinguishing  one  state  from  another  can  be  difficult  and  even  impossible,  and, 
similar  to  diagnosing  rheumatic  fever,  determining  a  state  of  a  given  system  must  be 
arbitrary  and  empirical.  Therefore,  developing  a  set  of  criteria  or  indicators  appropriately 
weighted  based  on  the  relative  likelihood  of  occurrence  of  a  system  state  may  also  be 
useful  in  performing  EBA.  The  methodology  presented  in  Chapter  3  develops  this  idea 
further,  describing  means  to  define  system  states  and  likely  indicators  of  those  states. 

2.7  Measures  and  Indicators 

The  key  question  of  EBA  asks  “How  do  assessors  know  if  the  effect  has  been 
achieved?”  Measures  and  indicators  are  the  tools  used  to  answer  this  question.  In  this 
thesis,  measures  denote  the  actual  numerical  value  associated  with  the  assessment.  This 
thesis  will  also  refer  to  measures  as  metrics.  Indicators  denote  the  event  that  is  to  be 
described  by  a  measure.  For  example,  an  indicator  of  an  effect  might  be  “the  enemy 
daily  sortie  rate  decreases.”  Then,  the  actual  number  or  percentage  decrease  of  sorties 
would  be  the  measure — say  “25%  decrease”  or  “30  sorties”  for  example.  Two  typical 
measures  in  EBO  and  assessment  literature  are  Measures  of  Performance  and  Measures 
of  Effectiveness.  Timmerman  adds  to  these  a  Measure  of  Interaction,  discussed  in 
Section  2.72.. 

2.7.1  Measures  of  Performance  and  Effectiveness. 

Two  types  of  measures  are  used  in  practice  to  aid  EBA — Measures  of 
Perfonnance  (MOPs)  and  Measures  of  Effectiveness  (MOEs).  MOPs  exist  to  track  task 
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accomplishment.  MOPs  serve  as  a  metric  for  the  level  of  completion  of  the  planned 
actions.  MOEs  exist  to  track  the  effects  of  actions  taken  on  the  enemy.  MOEs  measure 
changes  in  the  PMESII  system,  or  SoSA  model.  MOEs  focus  on  effects  achievement. 
MOEs  are  also  key  measures  of  progress  towards  the  change  in  system  behavior. 
Combined  together,  MOPs  and  MOEs  provide  an  assessment  of  current  operations 
performance  and  help  to  identify  trends  affecting  future  operations  (JWFC  Doctrine  Pam 
7,  2004: 12,  17).  This  thesis  will  discuss  methods  of  using  and  combining  MOPs  and 
MOEs  in  order  to  perform  campaign  assessment. 

2.7.2  Timmerman ’s  Three  Measures. 

Timmerman  defines  three  types  of  measures  using  Col  John  Boyd’s  Observer, 
Orient,  Decide,  Act  (OODA)  loop  model.  Timmerman’s  EBA  process  views  combat 
operations  as  two  interlocking  OODA  loops — one  for  BLUE  forces  and  one  for  RED 
forces  (Figure  2.7).  Timmerman’s  measures  for  EBA  follow  directly  from  this  model. 
The  first  of  Timmerman’s  measures  is  the  Measure  of  Effort.  This  corresponds  to  the 
MOP  defined  earlier.  As  Timmerman  puts  it,  “the  [C]FACC  has  to  know  the  present 
capabilities  of  his  forces  and  the  actions  they  are  currently  carrying  out.”  The  second  of 
Timmerman’s  measures  is  the  Measure  of  Interaction  (MOI).  An  MOI  is  the  measure  of 
“the  immediate  results  of  [coalition]  actions  and  their  interaction  with  enemy  efforts  and 
the  environment.”  Timmerman’s  third  measure  is  identical  to  the  MOE  previously 
defined.  An  MOE,  according  to  Timmerman,  measures  the  “emerging  consequences  of 
those  results  on  the  enemy’s  capabilities  and  decisions”  (Timmerman,  2003:32-34). 


37 


The  addition  of  MOIs  does  not  significantly  change  the  methodology  of  this 
thesis.  The  methodology  presented  in  Chapter  3  is  indicator  based.  Since  measures  of  all 
types  refer  to  the  quantitative  portions  of  the  indicators  used  in  this  thesis,  MOEs,  MOPs, 
and  MOIs  will  all  be  used,  though  not  explicitly  by  name.  MOPs,  however,  are  used  by 
name  when  tracking  the  progress  of  friendly  actions  taken  by  the  coalition  forces. 


Figure  2.7  Interacting  OODA  Loops  (Timmerman,  2003:33) 

Timmerman’s  description  of  assessing  stability  operations  differs  slightly  from 
his  description  of  assessing  combat  operations.  Timmerman  states  that  operations  that 
are  non-combat  in  nature  such  as  humanitarian  and  mobility  missions  can  be  viewed 
similarly  to  combat  operations  but  with  a  single  OODA  loop.  The  actions  in  stability 
operations  interact  only  with  the  environment.  Therefore,  Timmerman  asserts,  only 
Measures  of  Effort  and  Measures  of  Interaction  matter  since  there  is  no  opponent  whose 
decisions  are  being  affected  (Timmerman,  2003:39).  In  the  stability  operations  example 
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in  Chapter  4,  the  EBA  methodology  does  not  address  the  use  or  lack  of  use  of  any  type  of 


measure. 

2.7.3  Early  Warning  Indicators. 

Smith  states  that  assessing  effects  can  resemble  the  indications  and  warning 
intelligence  methods  developed  during  the  Cold  War  to  detennine  when  the  enemy  is 
preparing  an  attack.  The  methodology  Smith  describes  serves  as  a  skeleton  for  the 
methodology  presented  in  Chapter  3.  The  methodology  will  flesh  out  this  skeleton  to 
develop  and  apply  indicators  of  the  states  of  enemy  systems.  The  Cold  War  methodology 
Smith  describes  was  comprised  of  the  following  (Smith,  2002:382): 

■  development  of  extensive  list  of  indicators  based  on  postulated  actions  the  enemy 
might  take; 

■  indicators  are  made  into  intelligence  collection  priorities  and  regularly  tracked 
and  reported; 

■  observable  indicators  are  weighted  for  their  significance  and  for  the  place  they 
occupied  in  the  sequence  of  preparing  an  attack; 

■  weighted  indicators  are  aggregated  and  placed  into  an  algorithm  to  determine 
overall  probability  of  attack; 

■  a  fundamental  criterion  of  an  indicator  is  that  it  must  be  in  some  way  observable. 

2.7.4  Criteria  for  good  indicators. 

To  perform  EBA,  this  thesis  recognizes  the  necessity  of  developing  good 
indicators  of  the  effects  friendly  actions  have  on  the  enemy.  This  section  reviews  the 
criteria  of  indicators  that  can  serve  this  purpose. 
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A  good  indicator  is  one  that  provides  insight  to  the  state  of  an  enemy  system.  In 
order  to  provide  insight,  indicators  must  be  both  observable  and  relevant.  An  observable 
indicator  is  one  that  can  be  found,  fixed,  and  tracked  by  coalition  ISR  assets,  to  include 
ELINT,  HUMINT,  MASINT,  etc.  Indicators  may  also  be  observed  through  open  source 
means  of  intelligence.  Indicators  provide  the  data  for  any  EBA  model,  and  as  such  must 
be  observable  to  provide  any  input  into  a  model.  Put  simply,  analysts  cannot  assess  what 
they  cannot  see.  The  most  revealing  set  of  metrics  provides  no  value  if  no  observable 
data  exists  to  apply  to  the  metrics.  That  said,  there  are  certain  things  that  are  not  directly 
observable  such  as  the  internal  workings  of  the  enemy’s  mind  or  the  enemy’s  decision 
making  process.  In  order  to  detennine  the  state  of  the  system,  assessors  must  rely  on  the 
aspects  of  the  emerging  behavior  of  the  enemy  system  (Smith,  2002:383,  386). 

To  be  relevant,  an  indicator  must  connote  the  actions  taken  by  coalition  forces  and 
the  effects  those  actions  are  designed  to  achieve.  That  is,  an  indicator  must  be  connected 
via  a  causal  linkage  to  a  coalition  action  and  a  desired  effect.  In  as  much,  an  indicator 
should  reflect  the  state  of  the  system  after  the  coalition  action  has  been  executed. 
Similarly,  an  indicator  must  relate  somehow  to  the  system  in  order  to  serve  as  evidence 
for  a  state  change  of  the  system.  Though  coalition  assets  can  track  many  potential 
indicators  of  the  state  of  enemy  systems,  only  those  indicators  linked  to  coalition  action 
and  the  system  of  interest  can  provide  insight.  Therefore,  the  indicator  assessors  seek  is 
observable  evidence  of  a  behavioral  change  that  occurs  because  of  an  executed  coalition 
action  (Smith,  2002:383). 

Knowing  that  good  indicators  of  effects  must  be  both  observable  and  relevant,  the 
next  challenge  for  the  assessor  is  to  generate  a  set  of  indicators  that  meets  these  criterion. 
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Since  effects  represent  a  change  in  capability  or  behavior,  the  indicators  assessors  seek 
must  reveal  the  nature  of  the  system  capability  or  behavior.  Assessors  can  begin  by 
looking  at  performance  measures. 

Two  types  of  performance  measures  exist:  aggregate  indicators  and  point 
indicators.  Aggregate  indicators  provide  a  measure  of  the  overall  throughput  of  a  system. 
Point  indicators  provide  detection  and  measurement  of  an  event  that  differs  from  the 
established  nonn.  Both  aggregate  indicators  and  point  indicators  require  a  database  on 
the  system  in  question  and  continued  monitoring  of  that  system.  Assessors  can  use  these 
measures  to  reveal  how  the  system  functions.  Likewise,  physical  and  functional  effects 
can  indicate  the  success  of  a  particular  coalition  action.  Physical  and  functional  effects 
are  highly  quantifiable,  therefore  indicators  relating  to  system  capability  are  generally 
easier  to  detennine  (Smith,  2002:366,  369;  Mann,  Endersby,  and  Searle,  2002:37). 

Joint  Warfare  Center  Pamphlet  7  describes  how  indicators  should  represent 
changes  in  the  behavioral  state  of  a  system  (JWFC  Doctrine  Pam  7,  2004: 13).  Changes 
in  system  behavior  can  indicate  ongoing  adaptive  decision  making  processes  of  an 
adversary,  which  cross  over  into  the  psychological  realm  (Smith,  2002:370).  Mann  et  al 
note  that  psychological  changes  can  be  difficult  to  observe  and  track  accurately  (Mann, 
Endersby,  and  Searle,  2002:39).  However,  performance  assessment,  including  aggregate 
and  point  indicators,  can  be  used  to  assess  behavioral  changes.  Fundamentally, 
potentially  observable  behavior  indicators  fall  into  two  categories:  1)  evidence  of 
transmitting  guidance  for  a  course  of  action;  and  2)  the  physical  acts  that  the  course  of 
action  involves.  These  actions  of  will  likely  include  those  that  the  enemy  wants  the 
coalition  to  see  and  those  which  the  enemy  wants  to  conceal  (Smith,  2002:370,  388,  390). 
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Indicators  that  relate  directly  to  coalition  actions  and  to  planned  effects  are  not 
always  possible  to  observe  or  determine.  However,  other  areas  exist  to  determine 
appropriate  indicators  for  a  given  effect.  For  example,  any  military  reaction  by  an  enemy 
will  involve  decisions  in  the  political,  diplomatic,  and  economic  areas.  Similarly,  moves 
in  the  economic,  political,  and  diplomatic  areas  are  likely  to  parallel  actions  in  the 
military  area.  If  analysts  look  beyond  the  immediate  set  of  military  observables,  they 
might  be  able  to  find  indicators  that  are  substantially  easier  to  see  and  track  (Smith, 
2002:391).  In  as  much,  a  search  for  evidence  that  an  action  failed  to  achieve  an  effect 
may  be  more  productive  than  a  search  for  positively  reinforcing  evidence  (Mann, 
Endersby,  and  Searle,  2002:54).  Another  important  point  is  that  observations  need  not  be 
precise  to  be  useful.  It  may  be  sufficient  to  know  that  an  activity  has  intensified  or  that 
communications  between  certain  entities  has  occurred  in  order  to  detennine  a  change  in 
the  system  (Smith,  2002:390). 

This  thesis  advocates  the  use  of  indicators  as  a  means  to  detennine  the  state  of  the 
system  of  interest.  The  state  of  the  system  will  tell  assessors  how  close  coalition  forces 
are  to  achieving  the  desired  effects,  and  indicators  will  reveal  to  assessors  the  current 
state  of  the  system.  Highlighting  the  most  observable  indicators  of  success/failure  of  an 
effect  is  also  a  basis  for  an  ISR  collection  plan  (Smith,  2002:397). 

2.7.5  Assessing  Unexpected  Enemy  Reactions. 

The  methodology  presented  in  Chapter  3  leaves  room  to  perform  subjective 
assessments  when  the  enemy  reacts  in  a  manner  inconsistent  with  the  set  of  predefined 
indicators  for  evidence  of  a  desired  effect.  Likewise,  the  methodology  applies  a  progress 
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function  to  different  states  of  the  systems  of  interest.  The  effect  on  the  enemy  is  a 
function  of  the  combination  of  states  of  separate  systems.  The  combination  that  occurs  in 
reality  may  not  have  already  been  assigned  a  value  per  the  progress  function.  When  the 
enemy  reaction  is  unexpected  or  the  combination  of  system  states  has  not  been  assigned 
progress  value  a  priori,  the  indicators  and  states  witnessed  by  assessors  needs  to  be 
assessed  ad  hoc.  Smith  provides  assessors  with  a  list  of  relevant  questions  to  characterize 
unexpected  enemy  reactions,  system  states,  and  effects.  The  set  of  questions  helps 
assessors  to  identify  what  the  enemy  reaction,  system  state,  or  effect  is  and  the  nature  or 
extent  of  the  reaction,  state,  or  effect  (Smith,  2002:384): 

■  Is  the  reaction  symmetric/asymmetric? 

■  Does  the  reaction  involve  an  escalation  of  force? 

■  Was  there  no  observed  reaction? 

■  If  so,  does  this  represent  a  decision  not  to  act,  a  decision  to  postpone  any  reaction, 
an  inability  to  react,  or  an  inability  for  the  coalition  to  observe/detect  the  reaction? 

■  Which  military  capabilities  were  used? 

■  What  scale  and  geographic  scope? 

■  Which  warfare  areas? 

■  Was  it  a  lateral  escalation? 

■  Did  the  enemy  attempt  to  exploit  coalition  vulnerabilities?  If  so  which  ones? 

■  Did  the  reaction  expose  enemy  vulnerabilities,  if  so  which  ones? 

■  How  fast  did  the  enemy  react? 

■  How  long  was  the  reaction  sustained? 

■  How  long  could  it  have  been  sustained? 


43 


■  How  well  were  the  operations  coordinated/synchronized? 

■  What  was  observable  and  what  aspects  of  the  operation  may  have  been  unable  to 
be  detected? 

These  questions  are  used  to  detennine  how  the  enemy  perceived  the  military  actions  of 
the  coalition  forces  and  can  then  be  used  to  make  assessments  (Smith,  2002:384). 

2.8  Cause-and-Effect  Diagrams 

Developed  by  Dr.  Kaoru  Ishikawa  in  the  field  of  Quality  Control,  cause-and- 
effect  diagrams  are  a  powerful  tool  that  illustrate  the  relationship  between  results  of  a 
manufacturing  process  (characteristics)  and  the  possible  technical  causes  considered  to 
exact  an  effect  on  the  manufacturing  process.  Cause-and-effect  diagrams  are  drawn  to 
clearly  illustrate  various  causes  affecting  product  quality  by  sorting  out  and  relating  the 
causes.  Cause-and-effect  diagrams  are  sometimes  called  Ishikawa  diagrams,  or  fishbone 
charts  because  of  their  topology  (Ishikawa,  1991:229;  1989:25;  Ryan,  2000:23). 


Figure  2.8  Cause-and-Effect  Diagram  (Ishikawa,  1989:20) 
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Analysts  construct  cause-and-effect  diagrams  by  first  determining  the  quality 
characteristic  of  interest  (i.e.  wobble  during  machine  rotation).  This  characteristic  is 
something  that  one  wants  to  improve  and  control.  Next,  analysts  write  the  main  factors 
which  may  be  causing  the  quality  characteristic.  It  is  recommended  to  group  the  major 
possible  causal  factors  of  the  characteristic  into  such  items  as  raw  materials,  equipment, 
method  of  work,  measuring  method,  etc.  Each  individual  group  will  form  a  branch  of  the 
diagram  (Figure  2.8).  Finally,  onto  each  of  these  branches,  analysts  write  the  detailed 
factors  which  may  be  the  cause.  These  factors  are  like  twigs  on  the  branch.  On  each  of 
the  twigs,  analysts  write  even  more  detailed  factors  making  smaller  twigs.  Defining  and 
linking  these  causal  factors  should  lead  to  the  source  of  the  quality  characteristic 
(Ishikawa,  1989: 19-20).  See  Figure  2.9  for  a  small  example. 

2.6  microns 


Figure  2.9  Branching  of  Cause-and-Effect  Diagram  (Ishikawa,  1989:20) 

In  order  for  cause-and-effect  diagrams  to  be  useful,  they  need  to  illustrate  the 
appropriate  level  of  detail.  Cause-and-effect  diagrams  should  look  complicated  as  in 
Figure  2.10(a);  if  the  diagram  looks  more  like  Figure  2.10(b),  then  the  knowledge  of  the 
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process  is  too  shallow  or  the  diagram  is  too  generalized.  Furthermore,  if  the  cause-and- 
effect  diagram  only  lists  five  or  six  causes  (even  though  the  form  is  correct)  then  the 
diagram  is  probably  inadequate.  This  implies  that  a  useful  cause-and-effect  diagram  (one 
that  provides  insight  into  the  process)  requires  at  least  seven  to  ten  twigs  and  branches 
(Ishikawa,  1989:28-29). 

Cause-and-effect  diagrams  have  two  key  advantages:  1)  they  can  be  used  for  any 
problem  where  the  aim  is  to  get  results  by  knowing  the  relationships  between  causes  and 
effects;  2)  they  are  easily  understood  by  anyone  (Ishikawa,  1989:28-29).  This  thesis 
utilizes  cause-and-effect  to  develop  a  new  diagram  called  an  effects -tree  diagram. 
Cause-and-effect  diagrams  were  developed  to  actively  seek  out  causes  because  merely 
listing  causes  is  not  really  useful  (Ishikawa,  1989:25;  1991:231).  The  primary  interest  of 
this  thesis  is  effects.  The  author  notes  that  merely  listing  effects  is  not  really  useful 
either.  Therefore,  this  thesis  will  use  effects-tree  diagrams,  which  are  based  on  cause- 
and-effect  diagrams,  as  the  basis  for  a  tool  that  aids  EBO  planners  and  assessors  to 
actively  seek  out  effects  of  combat  operations. 


(a)  (b) 

Figure  2.10  Fishbone  Charts  (Ishikawa,  1989:28) 
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2.9  BLUE  FLAG  04 


This  thesis  will  also  build  upon  a  method  for  enumerating  effects  that  was 
developed  informally  at  a  BLUE  FLAG  exercise  in  February  of  2004  while  the  author 
served  as  the  Deputy  Chief  of  the  Operational  Assessments  (OA)  team,  12th  Air  Force. 

At  the  exercise,  the  CFACC  pushed  the  OA  team  to  develop  indicators  to  measure 
progress  towards  the  attainment  of  CFACC  objectives.  The  OA  team  worked  with  the 
Strategy  Division’s  planners  to  develop  these  indicators  by  enumerating  second  and 
third-order  (indirect)  effects  until  measurable  indicators  of  enemy  reactions  were  reached. 
This  method  for  enumerating  effects  was  based  on  single  and  multiple  actions  associated 
with  key  desired  effects  (Thoele,  2004).  In  Chapter  3,  this  thesis  takes  the  principles 
developed  by  the  OA  team  in  2004  and  combines  them  with  principles  from  the  cause- 
and-effect  diagrams  to  create  effects-tree  diagrams. 

2.10  40-70  Principle 

In  his  biography,  My  American  Journey,  Colin  Powell  describes  his  philosophy 
on  decision  making  with  incomplete  information.  He  believes  that  the  key  to  making 
good  decisions  is  to  not  make  decisions  too  quickly,  but  to  make  timely  decisions. 

Powell  states  that  in  to  order  make  an  informed  decision,  a  decision  maker  must  have  at 
least  40  percent  of  the  information.  Likewise,  a  decision  maker  should  not  wait  until  he 
has  100  percent  of  the  information,  because  by  then,  “it  is  almost  always  too  late” 
(Powell,  1995:393).  Powell  surmises  that  a  decision  maker  needs  between  40  to  70 
percent  of  the  information.  When  he  has  the  amount  of  information  in  that  range,  he  can 
“go  with  his  gut”  (Powell,  1995:393).  This  thesis  uses  the  lower  bound  of  Powell’s  range 
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of  required  information  as  a  thumb  rule  to  determine  the  state  of  a  system.  Phase  VI  of 
the  EBA  methodology  described  in  Chapter  3  will  require  evidence  for  at  least  40  percent 
of  the  indicators  for  a  given  state  before  assessors  can  reasonably  conclude  that  a  system 
is  in  that  state. 

2.11  Assumption  about  Enemy  Behavior 

In  order  to  determine  the  capability  and  behavior  states  of  the  enemy,  it  is 
necessary  to  anticipate  how  the  enemy  will  react  to  friendly  actions  taken  against  the 
enemy  PMESII  systems.  This  is  no  trivial  task.  As  Allen  points  out,  the  enemy  is 
“normally  uncooperative,  he  will  do  the  unexpected,  the  undesired,  and  the  unplanned” 
(Allen,  2005:8-9).  Similarly,  Smith  states  that  the  enemy  will  not  respond  as  expected; 
therefore,  the  planners  and  assessors  “must  plan  for  an  intelligent  adversary  who  will  be 
detennined  to  defeat  [commander’s]  efforts  by  whatever  means  possible”  (Smith, 
2002:254).  Smith  considers  this  particular  view  of  the  enemy  to  be  “prudent  planning” 
(Smith,  200:254).  In  this  thesis,  this  view  of  the  enemy  is  a  fundamental  assumption  that 
is  used  to  derive  potential  enemy  behaviors  resulting  from  friendly  actions.  To  be  clear, 
the  assumption  is  that  the  enemy  will  continue  to  fight  by  whatever  means  available  to 
him.  As  one  avenue  of  war-making  is  blocked,  the  enemy  will  seek  others,  detennined  to 
defeat  the  commander’s  efforts. 

2.12  Risk  Assessment 

In  his  Risk  Filtering,  Ranking  and  Management  (RFRM)  methodology,  Haimes 
provides  tools  useful  to  the  problem  of  performing  EBA.  RFRM  is  a  methodology  for 
identifying,  prioritizing,  assessing,  and  managing  scenarios  of  risk  to  large-scale  systems 
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from  multiple  overlapping  perspectives.  In  RFRM,  Haimes  integrates  empirical  and 
conceptual  methods,  descriptive  and  nonnative  methods,  as  well  as  quantitative  and 
qualitative  methods  to  implement  a  comprehensive  approach  to  the  problems  of 
identifying  and  managing  system  risks  (Haimes,  2004:277,279). 

Procedures  for  identifying  risk  require  analysts  to  establish  priorities  among  a 
large  number  of  individual  contributions  to  the  overall  system  risk.  To  manage  the  total 
risk  of  a  system,  identifying  what  can  go  wrong  and  the  associated  consequences  and 
likelihoods  (risk  assessment)  helps  generate  mitigation  options  with  their  trade-offs  and 
impacts  on  future  decisions.  RFRM  ranks  the  critical  elements  of  risk  and  thus 
contributes  to  the  analysis  of  mitigation  options,  which  facilitates  a  seemingly  intractable 
decision  problem  by  focusing  on  the  most  important  contributors  of  risk  (Haimes, 
2004:276-277). 

The  eight  phases  of  RFRM  are  summarized  as  follows  (Haimes,  2004:280): 

I.  Scenario  Identification — An  “as-planned”  or  “success  scenario”  is  developed 
using  Hierarchical  Holographic  Modeling  (HMM),  which  is  a  holistic 
methodology  to  decompose  the  attributes  of  a  system  while  representing  the 
system  through  multiple  perspectives  and  hierarchies;  risk  scenarios  are  then 
developed  based  on  what  can  go  wrong  in  the  success  scenario  (Haimes, 
2004:89,  280). 

II.  Scenario  Filtering — Risk  scenarios  are  filtered  based  on  the  responsibilities 
and  interests  of  the  relevant  system  user. 

III.  Bicriteria  Filtering  and  Ranking — The  remaining  risk  scenarios  are  further 
filtered  based  on  qualitative  likelihoods  and  consequences. 
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IV.  Multicriteria  Evaluation — Eleven  criteria  are  developed  that  relate  the  ability 
of  the  risk  scenario  to  defeat  the  system. 

V.  Quantitative  Ranking — Additional  filtering/ranking  of  scenarios  is 

accomplished  based  on  quantitative  and  qualitative  matrix  scales  of  likelihood 
and  consequence. 

VI.  Risk  Management — Risk  management  options  are  developed  to  mitigate 
remaining  risk  scenarios  including  cost  estimate,  performance  benefits,  and 
risk  reduction  of  each  option. 

VII.  Safeguarding  Against  Mission  Critical  Items — Analysts  evaluate  the 
performance  of  the  options  selected  in  phase  VI. 

VIII.  Operational  Feedback — Analysts  use  the  experience  and  information  gained 
during  implementation  of  mitigation  measures  to  refine  scenario  filtering  and 
the  decision  processes  of  earlier  phases. 

This  thesis  draws  from  two  general  aspects  of  RFRM  to  attack  the  EBA  problem.  First, 
as  Haimes  points  out,  the  phases  of  RFRM  “reflect  a  philosophical  approach  rather  than  a 
mechanical  methodology”  (Haimes,  2004:280).  The  RFRM  phases  imply  an  iterative 
process  wherein  analysts  move  in  and  out  of  phases,  repeating  each  as  necessary  to 
improve  the  overall  process.  The  EBA  phases  of  this  thesis  are  modeled  after  the  RFRM 
phases  to  capture  the  same  spirit  of  iteration.  Second,  by  incorporating  a  variety  of 
approaches — empirical  and  conceptual,  descriptive  and  normative,  as  well  as 
quantitative  and  qualitative — Haimes  is  able  to  capture  the  intent  of  the  decision  makers 
as  well  as  provide  mathematical  rigor  to  risk  assessment.  The  EBA  methodology 
presented  in  Chapter  3  aims  to  provide  this  same  balance  between  subjective  assessment 


50 


and  objective  assessment  in  order  to  provide  insight  to  the  completion  of  friendly  actions 
and  achievement  of  desired  effects. 

The  RFRM  contributes  a  more  specific  characteristic  to  this  thesis  as  well.  Phase 
III  of  RFRM  combines  two  types  of  information:  likelihood  of  what  can  go  wrong  with  a 
system,  and  the  associated  consequences.  To  do  this,  Haimes  presents  an  example  using 
the  ordinal  version  of  the  U.S.  Air  Force  risk  matrix  as  adapted  from  Military  Standard 
(MIL-STD)  882,  which  is  cited  in  Roland  and  Moriarty  (Haimes,  204:282).  Haimes’ 
matrix  is  presented  here  in  Figure  2.11. 


"''^Likelihood 

Effect 

Unlikely 

Seldom 

Occasional 

Likely 

A.  Loss  of  life/asset 
(Catastrophic  Event) 

B.  Loss  of  mission. 

C.  Loss  of  capability 
with  some  compromise 
to  mission. 

D.  Loss  of  some 
capability  with  no 
effect  of  mission. 

E.  Minor  or  no  effect. 

Frequent 


Low  Risk  Moderate  High  Risk  Extremely 
Risk  High  Risk 

Figure  2.11  Example  risk  matrix  for  RFRM  Phase  III  (Haimes,  2004:283) 


In  phase  III,  Haimes  combines  the  likelihoods  and  consequences  of  risk  scenarios 
into  a  joint  concept  called  “severity.”  To  do  this,  Haimes  divides  the  likelihood  of  risk 
source  into  five  discrete  ranges.  Similarly,  Haimes  divides  the  consequence  scale  into 
four  or  five  ranges.  The  scales  are  then  placed  in  matrix  form  and  the  cells  are  assigned 
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relative  levels  of  risk  severity.  The  scenario  categories  identified  in  phase  I  are  then 
distributed  into  cells  of  the  matrix;  those  falling  into  the  “low  risk”  category  are  filtered 
out  and  set  aside  for  later  consideration  (Haimes,  2004:282). 

This  thesis  utilizes  the  consequence  scale  that  Haimes  presents  in  phase  III  of 
RFRM  as  the  inspiration  for  an  ordinal  scale  for  the  state  of  enemy  systems.  After 
friendly  actions  have  been  taken  against  an  enemy  system,  the  system  will  transition  from 
its  current  state  to  another  state,  and  ultimately  to  a  desired  state.  The  EBA  methodology 
aims  to  track  the  system  as  it  transitions  from  state  to  state  using  the  states  of  the  ordinal 
scale  and  indicators  that  the  system  is  in  a  certain  state.  The  system  states  are  then  used 
to  determine  the  progress  of  commanders’  objectives. 

2,13  Value  Functions  and  Progress  Functions  in  EBO 

As  stated  earlier  in  the  chapter,  previous  assessment  efforts  have  used  linear 
additive  value  functions  to  detennine  attainment  of  commanders’  objectives.  This  thesis 
views  the  use  of  value  functions  as  inappropriate  for  the  purpose  of  assessing  attainment 
of  commanders’  objectives  in  an  EBO  context,  as  the  necessary  conditions  of  a  linear 
additive  value  function  are  not  met  within  the  interdependence  of  an  EBO  model. 

The  necessary  conditions  of  a  linear  additive  value  function  include  having 
attributes  of  an  alternative  that  are  mutually  preferentially  independent.  That  is,  all  other 
attributes  being  equal,  a  level  Xof  an  attribute  is  always  preferred  to  level  X'  (French, 
1986:  105-107,  120).  In  EBO  however,  this  condition  is  not  met.  If  the  attributes  are 
taken  to  be  the  enemy  systems,  and  the  attribute  levels  are  taken  to  be  the  states  of  those 
systems,  then  for  a  given  system,  an  arbitrary  state  X may  not  always  be  preferred  to  state 
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X ' .  When  other  systems  of  a  network  are  considered,  synergies  among  the  enemy 
systems  could  yield  situations  wherein  X '  is  preferred  to  X  when  the  states  of  other 
systems  are  considered.  For  example,  the  desired  end  state  of  a  conflict  may  include 
severing  the  command,  control,  and  communications  (C3)  links  to  an  enemy  facility. 
However,  if  the  lines  are  being  exploited  by  coalition  forces,  or  the  enemy  is  expected  to 
have  an  undesired  reaction  (say,  fire  theater  ballistic  missiles  if  communications  are  cut) 
then  keeping  the  C3  intact  may  actually  be  a  state  of  the  system  closer  to  the  attainment 
of  the  objective  than  when  the  C3  is  severed. 

Similarly,  in  previous  assessment  efforts  the  value  function  is  based  upon  a 
hierarchical  strategy-to-task  structure.  In  a  hierarchy  structure,  the  elements  within  each 
tier  of  the  hierarchy  are  divided  to  be  mutually  exclusive  and  collectively  exhaustive. 

The  elements  at  each  level  of  the  hierarchy  must  completely  describe  the  element  of  the 
next  higher  level  in  the  hierarchy  (collectively  exhaustive).  Likewise,  the  elements  on 
the  same  level  of  the  hierarchy  must  not  overlap  so  as  to  keep  from  double  counting  the 
element’s  contribution  to  the  level  above  it  (mutually  exclusive)  (Kirkwood,  1997:16-17). 
Chapter  3  will  discuss  in  detail  the  implications  of  these  two  properties  of  hierarchies  in 
an  EBO  model.  Essentially,  a  set  of  effects  are  defined  to  be  collectively  exhaustive  with 
regards  to  the  attainment  of  an  objective.  However,  since  an  effect  may  contribute  to 
multiple  objectives,  and  effects  can  contribute  to  achieving  other  effects,  the  effects  are 
inherently  interrelated,  thus  not  mutually  exclusive.  Violating  the  mutual-exclusivity 
property  also  makes  the  use  of  a  linear  additive  value  function  inappropriate  for  use  in  an 
EBO  model. 
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Instead  of  the  linear  additive  value  function,  this  thesis  will  introduce  a  progress 
function  in  Chapter  3  to  determine  attainment  of  objectives.  In  this  function,  the  different 
states  of  enemy  systems  will  be  considered  as  a  vector  of  states.  Military  planners  and 
subject  matter  experts  will  use  their  experience  and  intuition  to  map  these  vectors  to  a 
value,  which  determines  the  progress  towards  objective  attainment.  In  this  function,  the 
differences  between  the  state  vectors  are  best  expressed  qualitatively  rather  than 
quantitatively.  The  use  of  a  progress  function  (rather  than  a  value  function)  indicates  a 
subtle  difference  in  the  underlying  purpose  for  using  the  function.  A  value  function 
represents  the  preference  of  a  decision  maker  amongst  a  set  of  alternatives.  The  progress 
function  defined  in  this  thesis  represents  the  level  attainment  of  a  military  objective. 
While  these  two  functions  can  be  expressed  by  the  same  mathematical  formulations,  the 
functions  express  different  concepts.  This  thesis  aims  to  represent  the  state  of  an  enemy 
network  that  best  represents  the  commander’s  view  of  objective  attainment,  which 
implies  more  than  the  commander’s  preference  amongst  a  set  of  alternatives. 

2.14  Conclusion 

Chapter  2  presented  an  overview  of  EBO  that  includes  the  definition  of  effect — a 
change  in  the  state  of  system.  EBO  has  four  key,  interrelated  components:  a  System-of 
Systems  Analysis  (SoSA),  Effects-Based  Planning  (EBP),  Effects-Based  Execution 
(EBE),  and  Effects-Based  Assessment  (EBA).  To  develop  a  methodology  to  perform 
EBA,  this  thesis  draws  from  previous  assessment  efforts  such  as  the  large-scale  effort  of 
OIF.  In  order  to  improve  upon  past  efforts,  this  thesis  looks  to  medical  diagnosis 
methods  that  use  criteria  or  indicators  to  detennine  the  state  of  a  patient  or  system.  To 
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develop  the  methodology  further,  tools  in  the  fields  of  quality  control  and  risk  assessment 
are  used  to  derive  system  states  and  a  set  of  indicators  to  signal  the  presence  of  those 
states.  These  elements  are  then  combined  with  the  mathematics  from  decision  theory  to 
develop  progress  functions  that  represent  the  level  of  attainment  of  commanders’ 
objectives.  Chapter  3  presents  the  EBA  methodology  of  this  thesis  in  detail. 
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3  Methodology 


3.1  Introduction 

The  ultimate  goal  of  Effects-Based  Assessment  (EBA)  is  to  provide  insight  to  the 
Combined  Forces  Air  Component  Commander  (CFACC)  and  Combined  Air  Operations 
Center  (CAOC)  staff.  The  CFACC  wants  to  know  what  progress  has  been  made  towards 
his  objectives  and  desired  effects.  Similarly,  the  CAOC  staff  wants  to  know  how  the 
operational  plan  is  doing  with  regards  to  achieving  the  desired  effects,  and  just  as 
important,  the  CAOC  staff  wants  to  know  how  coalition  forces  are  doing  with  regards  to 
executing  the  plan.  The  EBA  methodology  described  in  this  chapter  is  designed  to 
provide  this  insight. 

Section  3.2  describes  the  mathematical  construct  of  EBA.  The  progress  function 
is  defined  including  the  use  of  effects,  indicators,  regions  and  state  vectors.  Section  3.3 
describes  a  seven-phase  methodology  for  implementing  EBA  in  an  operational  context. 
The  methodology  begins  with  defining  effects  and  concludes  with  a  method  to  view  the 
progress  of  coalition  actions  and  their  effects  on  the  enemy  over  time. 

3.2  Mathematical  Construct  of  EBA 

In  general  the  EBA  process  aims  to  provide  a  quantitative  measure  of  progress  of 
the  CFACC  objectives  and  desired  effects.  Per  the  Effects-Based  Planning  (EBP)  Course 
of  Action  (COA)  development  process,  the  progress  towards  effects  achievement 
determines  the  progress  towards  objective  attainment.  That  is,  the  progress  of  the 
objectives  is  a  function  of  the  progress  of  the  effects.  Similarly,  the  progress  of  the 
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effects  is  a  function  of  friendly  actions  and  enemy  reactions.  The  EBA  methodology 
tracks  friendly  actions  and  observes  the  enemy  reactions  in  order  to  assess  the  progress  of 
the  desired  effects.  The  progress  of  the  effects  is  then  used  to  generate  the  progress  of  the 
CFACC  objectives. 

3.2.1  Progress  of  an  Objective. 

For  each  CFACC  objective,  the  EBP  process  produces  a  finite  set  of  desired 
effects.  Fet  Ek  =  {el,e2,...,e  }  be  the  set  of  nk  desired  effects  for  objective  k.  For  each 

system  q,  there  exists  a  set  of  states  S  =  {s0^1,...^0},  where  q  =  1,2 ,...,nk  .  This 

notation  assumes  one  system  for  each  desired  effect,  but  the  notation  could  be  adapted  to 
accommodate  vectors  of  multiple  systems  per  effect. 

In  general,  an  effect  is  a  change  in  system  state  As  where 

As  =  sw^>sx  for  sw,  sx  eSq 

where  sw  —>  sx  denotes  the  transition  from  state  5"  to  state  sx.  In  order  to  achieve  desired 
effect  eeEk  ,  system  q  must  transition  from  the  initial  state  s°  to  the  desired  state  ,s,c' 
denoted  as 

e  =  As  =  s°  — >  se 

C,  ~  ^  q  bq  ^  \ 

That  is,  the  desired  effect  on  system  q  is  the  change  from  the  initial  state  s°  to  the  desired 
state  seq .  Achievement  of  these  effects  will  be  used  to  determine  the  progress  towards  the 
attainment  of  objective  k. 

This  thesis  defines  progress  as  the  level  of  attainment  of  an  objective,  represented 
by  a  value  p  e  [0, 1] ,  where  0  denotes  no  attainment  and  1  denotes  complete  attainment. 
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Let  Ok  be  the  progress  of  objective  k.  Then,  Ok  can  be  described  as  a  function  of  the 


current  system  states: 

0k  =  /0  VS2-’S„t) 

where  /  is  a  progress  function  such  that  / :  M"  — »  [0, 1]  and  sq  is  the  state  of  system  q. 
Using  this  function,  Ok  =  f(sfse2...,senf  =  \  and  Ok .  =  f  (s°,s°...,s°t  )  =  0 . 

In  the  previous  assessment  efforts  mentioned  in  Chapter  2,  (reference  Sections  2.5 
and  2.13),  the  progress  of  objectives  was  determined  by  an  additive  value  function. 
Additive  value  functions  require  two  fundamental  assumptions  to  be  met:  1)  the 
components  within  each  level  of  the  hierarchy  are  mutually  exclusive  with  regards  to 
each  other  and  2)  the  components  are  collectively  exhaustive  with  regards  to  the  next 
level  up  the  hierarchy.  EBO  considers  operational  effects  that  are  inherently  dependent 
and  therefore  not  mutually  exclusive.  Achievement  of  one  effect  may  contribute  to  or 
impede  the  achievement  of  other  effects.  Therefore,  the  first  assumption  is  usually  not 
met  and  the  use  of  an  additive  value  function  is  generally  inappropriate.  However,  in 
EBO  the  effects  are  taken  to  be  collectively  exhaustive  within  a  given  objective.  In  the 
sense  of  the  progress  function,  collectively  exhaustive  means  that  the  set  of  effects 
provides  the  complete  set  of  arguments  for  the  progress  function. 

The  vector  sA  =  (s™ ,s” )  e  is  called  the  state  vector  of  objective  k, 

where  s'"  represents  the  current  state  of  system  q,  and  m  e  {0,l,...,e}.  Each  state  vector 
s*.  represents  a  different  point  on  the  ///-dimensional  hypercube,  which  represents  the 
progress  towards  attainment  of  objective  k.  For  the  system  of  interest  n[q],  as  the  state  s 
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of  n[q]  changes,  the  element  s'”  changes  and  sk  moves  from  point  to  point  on  the 
hypercube  progressing  towards  the  attainment  of  objective  k. 

Consider  the  case  where  nk  =  2  and  each  system  has  five  states.  All  systems 

begin  in  state  0  (Figure  3.1a).  After  coalition  actions  have  been  taken,  the  objective 
moves  to  point  (2,2)  (Figure  3.1b).  Put  another  way,  the  system  of  interest  in  the  first 
effect  n^ ,  which  corresponds  to  the  first  element  of  the  vector,  transitions  to  state  2. 

Likewise,  the  system  of  interest  in  the  second  effect  n^  ,  which  corresponds  to  the 

second  element  of  the  vector,  also  transitions  to  state  2.  In  the  next  assessment,  the 
systems  transitions  to  state  vector  sk  =  (3, 1)  as  shown  in  Figure  3.1c.  In  Figure  3 . 1  d,  the 

systems  have  transitioned  to  state  vectors  (3,4)  and  then  to  its  desired  end  state  (5,5). 
The  progress  function  / (•)  can  be  viewed  as  mapping  the  state  vectors  to 

bounded  regions  of  the  /(-dimensional  hypercube.  Let  R"  cf,  veN  be  a  region  of 

state  vectors  sk  such  that  all  sk  e  R  map  to  the  same  progress  value  py.  Then,  0*  =  pv. 

Figure  3.2  depicts  an  example  of  three  regions  (y  =  1,  2,  3)  and  their  associated  progress 
values.  In  this  example,  the  first  state  vector  (0,0)  is  in  Region  1 .  The  second  state 
vector  (2,2)  is  in  Region  2.  The  third  state  vector  (3,1)  returns  to  Region  1.  The  fourth 
and  fifth  state  vectors  (3,4)  and  (5,5)  are  in  Region  3.  Therefore,  the  values  for  Ok  for 
each  assessment  corresponding  to  the  state  vectors  in  Figure  3.2  are  as  follows:  0.33, 
0.67,  0.33,  1.00,  and  1.00.  Note  that,  by  definition  of  Region  3,  the  desired  end  state 
vector  si.  ={(3,3);  (4,3);  (5,3);  (3,4);  (4,4);  (5,4);  (3,5);  (4,5);  (5,5)} . 
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Figure  3.1  Progress  of  Objective  k 
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Figure  3.2  Progress  regions  and  /^.-values. 


3.2.2  Relationship  between  System  States  and  Assessment  Indicators. 

For  each  state  s"'  of  system  n[q],  s'"  is  defined  by  a  function  g  :  I  Sq  such  that 

s”  =  g(.h,h,-,ir) 

where  /  =  {ivi2...,ir}  is  a  set  of  indicators  i,  i  e  {0,1}  .  When  a  predefined  number  of 
indicators  exists  for  a  given  state,  i  =  1  for  some  i  e  I ,  n[q]  is  said  to  be  in  state  s”  .  In 
Figure  3.3,  the  states  are  defined  such  that  the  system  will  transition  in  a  series  from  the 
initial  state  5°  to  the  desired  state  se  as  a  result  of  the  friendly  actions  taken  and  enemy 
reactions. 

Using  indicators  to  assess  the  state  of  the  system  of  interest,  assessors  then  use  the 
changes  in  system  states  (the  effects)  to  build  the  state  vectors.  The  state  vectors  are  then 
used  as  inputs  to  the  progress  function.  Thus,  this  EBA  construct  builds  upon  the 
indicators  to  make  an  assessment  of  the  progress  towards  objective  attainment.  In  order 
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to  perform  these  calculations,  all  the  relevant  parameters  must  be  defined  (i.e.,  the  states 
of  the  in  systems  for  objective  k,  or  the  state  vector  (s”'1  ,s"h  ,—,s™nk  )•  The  remainder  of 

the  chapter  describes  how  the  planners  and  assessors  develop  the  parameters  beginning 
with  the  central  piece  of  EBO,  the  effects. 


Figure  3.3  State  transition  diagram  for  system  n[q] 

3.3  Seven-Phase  EBA  Methodology 

In  order  to  apply  the  above  theoretical  construct  to  an  operational  environment, 
this  thesis  decomposes  the  EBA  process  into  seven  phases.  At  any  point  in  the  process, 
planners  and  assessors  can  be  simultaneously  working  in  any  of  the  phases.  The  phases 
have  a  general  chronological  order  with  each  phase  producing  material  required  in  the 
subsequent  phases.  However,  unlike  a  rigid  series  of  fixed  steps,  the  phases  can  be 
revisited  iteratively  and  in  various  orders  to  improve  the  EBA  process  and  output. 

EBA  is  a  vital  and  complementary  part  of  EBP.  Planners  and  assessors  work  in 
concert  to  develop  an  assessable  effects-based  plan.  The  EBA  and  EBP  processes 
support  and  enhance  each  other;  as  the  plan  changes,  the  assessment  plan  also  changes. 
Just  as  in  EBP,  much  of  the  work  described  in  the  EBA  phases  should  be  accomplished 
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prior  to  the  start  of  the  Effects-Based  Execution  (EBE).  During  combat  operations,  the 
elements  of  EBP  and  EBA  are  then  subsequently  altered  as  the  campaign  evolves. 

The  phases  of  EBA  are  as  follows: 

Phase  I:  Desired  Effect  Definition.  For  each  operational-level  effect,  assessors 
and  planners  develop  a  statement  completely  describing  the  desired  effect;  this 
statement  identifies  the  enemy  system  of  interest,  the  initial  state  of  the  system, 
the  desired  end  state  of  the  system,  and  the  start  and  end  times  of  the  state  change. 
Phase  II:  Indicator  Development.  Planners  and  assessors  develop  a  set  of 
measurable  indicators  based  on  anticipated  enemy  system  reactions  to  planned 
coalition  actions  and  desired  effects.  This  thesis  defines  a  measurable  indicator  as 
one  that  is  observable  and  distinguishable.  Effects-tree  diagrams  are  introduced 
to  facilitate  this  phase. 

Phase  IIP  System  State  Definition.  The  set  of  measurable  indicators  is 
partitioned  into  ordinal  states  that  describe  the  system  of  interest  as  it  reacts  to 
friendly  actions. 

Phase  IV:  Actions  Assessment.  Coalition  actions  are  tracked  using  Measures  of 
Performance  (MOP),  similar  to  those  used  in  OIF,  to  track  the  CFACC  actions. 
The  MOPs  are  then  plotted  on  a  time  axis  to  view  trends. 

Phase  V:  Effects  Assessment.  The  indicators  and  MOPs  are  used  as  evidence  for 
an  assessment  of  the  state  of  the  systems  of  interest.  The  resulting  state  of  the 
systems  is  monitored  over  time  to  view  trends  towards  the  desired  end  state. 

Phase  VI:  Objectives  Assessment.  The  progress  function  is  defined  by  planners 
anticipating  state  vectors  that  will  result  from  friendly  actions.  As  state  vectors 
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are  determined  during  EBE,  the  vectors  are  mapped  to  progress  values  according 
to  this  function. 

Phase  VII:  Campaign  Assessment.  Actions  assessment,  effects  assessment,  and 
objective  assessment  are  brought  together  with  the  time  dimension  to  observe 
how  the  enemy  systems  react  to  the  friendly  actions.  In  this  phase,  assessors 
determine  if  the  actions  executed  are  achieving  the  desired  effects  and  are 
accomplishing  the  CFACC  objectives  as  planned. 

3.3.1  Phase  I:  Precisely  Defining  Effects  to  be  Assessable. 

Effects  are  the  heart  of  EBO,  and  desired  effects  of  friendly  actions  on  the  enemy 
are  what  direct  the  entire  EBO  process.  Effects  direct  the  EBA  portion  of  the  process  in 
two  ways:  1)  assessing  the  attainment  of  commander’s  objectives  is  accomplished  by 
assessing  the  achievement  of  a  set  of  desired  effects;  2)  effects  are  used  to  derive 
indicators  that  determine  the  achievement  of  those  effects.  Phase  I  lays  out  the 
specifications  required  of  an  effect  definition  so  that  the  achievement  of  the  effect  can  be 
accurately  assessed. 

The  EBA  process  begins  with  a  set  of  CFACC  objectives  and  a  set  of  desired 
effects  that  must  be  achieved  in  order  to  attain  the  objectives.  For  each  objective, 
planners  identify  a  set  of  effects  that  apply  to  that  objective.  With  regards  to  the 
overarching  objective,  the  set  of  effects  must  be  collectively  exhaustive.  Put  another 
way,  for  a  given  objective,  there  should  exist  an  effect  for  each  system  that  must  undergo 
a  state  change.  These  effects  taken  together  completely  describe  everything  that  must 
occur  to  attain  the  objective.  The  collectively  exhaustive  property  enables  the  analytical 
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leap  from  effects  achievement  to  objective  attainment;  if  all  the  effects  have  been 
achieved,  then  the  objective  has  been  attained. 

In  order  to  assess  the  achievement  of  the  effects,  assessors  must  derive  from  the 
effects  measurable  indicators  that  give  insight  to  the  state  of  the  system.  Indicators  are 
evidence  that  a  system  state  change  has  occurred.  A  well-defined  effect  description 
guides  the  search  for  indicators  so  that  assessors  can  track  data  representing  the  resulting 
system  states  after  friendly  actions  have  been  executed.  In  order  to  meet  these 
assessment  requirements,  the  effects  must  be  clearly  and  precisely  defined. 

Gallagher  et  al  state  that  an  effect  is  clearly  and  precisely  defined  when  a  desired 
functional  capability  or  desired  behavior  along  with  a  range,  extent,  start  time  and  end 
time  are  specified  (Gallagher,  True,  and  Whiteman,  2004:9).  This  thesis  adds  to  these 
specifications  the  initial  state  of  the  system,  the  system  state  before  any  coalition  actions 
are  taken.  Therefore,  for  each  effect,  planners  and  assessors  need  to  define  the  following 
specifications: 

■  System  of  interest ;  this  is  what  Gallagher  et  al  describe  as  the  range;  a  capability- 
range  is  the  affected  area,  such  as  target,  city,  region  or  state,  whereas  behavior- 
range  is  the  affected  individual,  group  or  nation  (Gallagher,  True,  and  Whiteman, 
2004:9);  this  specification  answers  the  question  “What  system  is  to  be  affected?” 

■  Initial  State  of  the  System ;  this  specification  is  the  baseline  for  comparison  after 
coalition  actions  have  been  taken.  This  specification  answers  the  question  “What 
is  the  beginning  state  of  the  system?”  To  answer  this  question,  enemy  systems 
need  to  be  tracked  before  conflict  ensues.  Assessors  may  need  to  integrate 
historical  data  to  establish  performance  norms  of  physical  systems  or  behavioral 
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norms  of  human  systems  (Smith,  2002:397).  A  quality  SoSA  model,  should  be 
able  to  provide  input  as  to  the  initial  states  of  the  system  as  well. 

■  Desired  End  State',  this  is  what  Gallagher  et  al  describe  as  the  Extent;  the  extent 
specifies  the  resulting  level  of  capability  or  behavior — whether  it  is  decreased, 
maintained  or  increased  (Gallagher,  True,  and  Whiteman,  2004:9);  this 
specification  answers  the  question  “What  is  the  desired  change  in 
capability/behavior  of  the  system?’ 

■  Start  time;  start  time  answers  the  question  “When  is  the  change  to  take  place?” 

■  End  time;  end  time,  when  combined  with  start  time,  answers  the  question  “How 
long  does  the  effect  last?” 

In  conclusion,  effects  are  well-defined  when  they  are  described  by  the  system  of 
interest,  the  initial  state  of  the  system,  the  desired  end  state,  and  start  time  and  duration  of 
the  effect.  Effects  defined  this  way  will  adequately  support  the  assessment  of  the 
objective  and  the  derivation  of  appropriate  indicators. 

3.3.2  Phase  II:  Developing  Measurable  Indicators  with  Effects-Tree 
Diagrams. 

The  hardest  task  of  an  assessment  effort  is  detennining  what  data  is  observable, 
obtainable  by  intelligence  assets,  and  provides  insight  to  the  nature  of  the  enemy’s 
capabilities  and  behavior.  How  do  analysts  know  if  the  effects  are  being  achieved?  What 
indicators  should  be  chosen?  What  metrics  should  be  used?  What  intelligence  options 
are  available  for  data  collection  on  the  indicators?  How  can  more  options  be  generated? 
One  approach  to  determine  the  state  of  an  enemy  system  (as  discussed  in  Section  2.9) 
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includes  an  analysis  of  the  most  likely  enemy  reactions  to  friendly  actions  taken  against 
enemy  system  nodes.  Phase  III  builds  on  this  approach  to  generate  a  set  of  measurable 
indicators.  This  thesis  defines  a  measurable  indicator  as  one  that  is  observable  and 
distinguishable.  Observable  denotes  the  indicator  is  detectable  by  the  intelligence  assets 
available  to  the  JFC,  including  but  not  limited  to  traditional  ISR,  open-source 
intelligence,  SIGINT,  HUMINT,  ELINT,  MASINT,  etc.  Distinguishable  denotes  that 
when  the  indicator  changes,  the  change  is  evident  to  the  assessors.  For  example,  when 
assessing  the  combat  strength  of  an  enemy  infantry  division,  the  unit  itself  may  be 
observable.  However,  the  particular  strength  of  the  unit  may  not  be  distinguishable; 
similarly,  the  unit  may  not  actually  be  distinguishable  from  other  enemy  units. 
Conversely,  the  life  status  of  an  enemy  leader  (whether  he  is  living  or  dead)  is  certainly 
distinguishable,  but  it  may  not  be  observable  because  his  whereabouts  are  unknown. 

A  difficult  task  for  planners  is  determining  the  most  likely  effects  to  be  generated 
by  different  courses  of  action.  For  instance,  destruction  of  a  single  node  can 
simultaneously  have  indirect  effects  at  all  levels  of  warfare  and  across  all  types  PMESII 
systems.  With  multiple  layers  of  indirect  effects,  it  becomes  increasingly  difficult  to 
precisely  predict  enemy  system  reactions.  Historically,  its  been  difficult  to  predict 
beyond  third-order  effects  with  any  certainty  (Smith,  2002:397;  Mann,  Endersby,  and 
Searle,  2002:34,  39).  To  overcome  these  challenges,  this  thesis  introduces  a  tool  called 
an  effects-tree  diagram.  This  section  will  first  describe  the  general  fonn  and  construction 
of  an  effects-tree  diagram  and  then  follow  with  a  detailed  example. 
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3 .3 .2 . 1  Constructing  Effects-  Tree  Diagrams. 


Developing  an  effects-tree  diagram  is  similar  to  creating  a  cause-and-effect 
diagram,  the  quality  control  tool  described  in  Section  2.8.  It  is  necessary  to  know  both 
causes  and  effects  in  great  detail  and  in  concrete  terms  in  order  to  illustrate  their 
relationship  and  make  them  useful  (Ishikawa,  1989:18).  Effects-tree  diagrams  can  be 
thought  of  as  the  reversal  of  cause-and-effect  diagrams.  Cause-and-effects  diagrams 
begin  with  a  resulting  effect  and  work  backward  to  detennine  root  causes.  Effects-tree 
diagrams  begin  with  a  military  action  against  an  enemy  network  node  (a  cause)  and  work 
forward  to  determine  the  resulting  effects. 

Step  1  (Figure  3.4):  Determine  the  operational-level  enemy  system  of  interest  to 
be  diagramed.  Write  the  initial  state  5°  on  the  left  side  of  the  diagram  and  draw  a  box 
around  it.  Similarly,  write  the  desired  state  se  on  the  right  side  of  the  diagram  and  draw  a 
box  around  it.  Then  draw  an  arrow  from  the  initial  state  to  the  desired  state.  This  arrow 
represents  the  transition  of  the  system  from  the  initial  state  to  the  desired  state. 


Figure  3.4  Effects-Tree  Diagram  Step  1 

Step  2  (Figure  3.5):  List  any  indicators  of  the  initial  and  desired  state  that  can  be 
derived  from  the  effect  definition  in  Phase  I.  Add  these  to  the  diagram  by  drawing  a 
branch  from  the  system  state.  On  this  branch  draw  smaller  branches  containing  the 
indicators.  Indicator  branches  are  denoted  by  the  circles  on  the  branch.  It  may  be 
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necessary  to  list  indirect  effects  of  the  system  state  in  order  to  reach  measurable 
indicators  of  the  system  state. 


Figure  3.5  Effects-Tree  Diagram  Step  2 

Step  3  (Figure  3.6):  Draw  the  planned  friendly  actions  Al,A2,...,An ,  to  be  taken 

against  the  system  with  arrows  directed  to  the  shaft  of  the  central  arrow.  These  actions 
are  prescribed  by  the  O-E-N-A-R  chain  resulting  from  EBP.  In  effects-tree  diagrams, 
these  arrows  represent  causal  linkages. 


Figure  3.6  Effects-Tree  Diagram  Step  3 
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Step  4:  For  each  action,  draw  a  branch  of  arrows  down  from  the  central  line.  The 
vertical  line  of  this  branch  represents  the  set  of  all  indirect  effects  Er  resulting  from  the 
achievement  of  the  desired  end  state  se.  The  arrows  represent  enemy  reactions  (indirect 
effects  Ei)  to  the  friendly  action  that  resulted  in  the  desired  end  state  of  the  system. 

Assessors  now  consider  the  indirect  effects  that  would  be  caused  by  each  action 
taken  in  isolation.  That  is,  assume  that  action  A„  is  100-percent  successful  at  achieving 
the  desired  effect  on  the  enemy  system.  Assume  no  other  friendly  actions  were  taken, 
and  the  enemy  system  has  transitioned  to  the  desired  end  state  se.  This  is  done  for  each 
action,  hence  a  branch  is  drawn  for  the  set  of  enemy  reactions  resulting  from  the  desired 
state  caused  by  each  action  as  shown  in  Figure  3.7.  Assessors  should  consider  the 
reaction  of  all  nodes  within  the  system  as  well  as  the  reactions  of  other  systems  of  the 
PMESII  network  connected  to  the  system  of  interest.  To  consider  all  potential  enemy 
reactions,  assessors  assume  that  the  enemy  will  continue  to  fight  by  all  means  available  to 
him  (reference  Section  2. 1 1).  With  this  assumption,  an  additional  question  for  the 
assessors  becomes  “Given  that  action  An  alone  causes  the  desired  end  state  se,  how  does 
the  enemy  react  in  order  to  keep  fighting?”  The  answers  to  this  question  are  then  added 
to  the  diagram  as  arrow  branches. 
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Figure  3.7  Effects-Tree  Diagram  Step  4 

Step  5  (Figure  3.8):  Adding  branches  to  the  diagram  should  lead  to  measurable 
indicators  of  the  indirect  effects  ivi2,...,ir ;  therefore  assessors  should  continue  adding 

branches  until  they  reach  measurable  indicators.  Assessors  must  try  to  understand  the 
cause-and-effect  relationship  as  fully  as  possible  and  increase  the  number  of  sub¬ 
branches  by  continually  asking,  “how  does  that  action  affect  the  system?  How  would  the 
system  react?  How  can  the  effect  be  observed?”  Planners  and  assessors  must  ask  what 
else  might  happen,  because  in  a  complex  system,  no  action  ever  creates  a  single  outcome. 
There  is  always  some  other  indirect  effect  to  consider  (Mann,  Endersby,  and  Searle, 
2002:52).  It  is  important  to  note  that  to  arrive  at  measurable  indicators,  several 
intennediate  arrow  branches  (indirect  effects)  may  need  to  be  drawn  before  drawing  the 
indicator  branch. 
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The  effects-tree  diagram  helps  organize  and  relate  the  impact  of  actions  and  their 
indirect  effects  to  the  rest  of  the  enemy  system.  The  key  to  observing  indirect  effects  in 
effects-tree  diagrams  is  the  reiteration  of  the  questions  How  does  that  action  affect  the 
system?  How  would  the  system  react?  How  can  the  effect  be  observed?  For  each 
indirect  effect  generated,  analysts  need  to  expand  the  tree  by  answering  these  questions. 

For  planning  purposes,  assessors  should  include  as  many  relevant  people  as 
possible  when  building  effects-tree  diagrams.  Planners,  assessors,  and  intelligence 
analysts  should  all  be  included  in  order  to  determine  enemy  reactions  and  indirect  effects 
of  an  action. 

Assessors  need  to  make  an  effects-tree  diagram  for  each  effect  within  a  given 
objective.  Likewise,  when  the  effects-tree  diagram  begins  to  reveal  potential  indicators, 
it  is  useful  to  classify  the  indicators  by  organizations  that  can  provide  the  necessary  data 
to  perfonn  the  assessments  process.  To  this  end,  do  not  cut  effects  and  potential 
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indicators  because  there  is  no  known  way  to  observe  or  quantify  them.  Include  all  effects 
considered  important  for  the  action  and  desired  effect,  regardless  of  whether  or  not  they 
are  presently  being  measured  or  are  capable  of  being  measured;  these  effects  may  still 
provide  insight  to  planners  after  the  measurable  indicators  have  been  chosen. 

When  the  diagram  is  complete,  planners  and  assessors  should  have  developed 
approximately  10  potential  indicators  for  each  desired  effect.  A  set  of  indicators  of  this 
size  will  enable  partitioning  the  indicators  into  states  in  Phase  III.  Keep  in  mind, 
however,  that  in  practice,  the  total  number  of  indicators  capable  of  being  tracked  for  an 
entire  objective  is  around  50  to  60.  Note,  a  single  indicator  may  provide  evidence  for 
multiple  effects  and  should  be  used  for  as  many  effects  as  appropriate.  These  indicators 
then  need  to  be  coordinated  with  the  ISR  division  in  order  to  make  the  indicators  part  of 
the  overall  ISR  collection  plan. 

3.3.2.2  Effects-Tree  Diagram  Example. 

The  following  example  of  an  effects-tree  diagram  is  based  on  a  scenario  of  the 
notional  country  of  Exstan.  As  Exstani  governance  moves  towards  a  stable  democracy, 
the  next  round  of  elections  are  planned  to  take  place  in  six  months.  As  the  elections  get 
closer,  terrorist  attacks  have  increased  in  number  and  intensity.  The  coalition  has 
determined  the  following  desired  effect  against  Kobra,  a  ruthless  terrorist  organization 
determined  to  rule  Exstan: 

■  System  of  interest  Kobra  terrorist  organization. 

■  Initial  State  of  the  System :  3  Kobra  Commander  hideouts;  4  Kobra  training 
camps;  10  other  terror  group  bases  sympathetic  to  Kobra;  15  air  bases  and 
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military  storage  facilities.  Kobra  is  capable  of  conducting  as  many  as  ten 
simultaneous  attacks  throughout  the  country  with  varying  degrees  of  destruction 
and  casualties. 

■  Desired  End  State :  Kobra  incapable  to  perform  a  single  attack  that  results  in  5  or 
more  casualties. 

■  Start  time:  Effect  to  begin  on  or  before  D+l 5. 

■  End  time :  Effect  not  to  end  before  D+l 95. 

The  first  step  of  the  tree-diagram  flows  directly  from  the  desired  effect  definition  as  seen 
in  Figure  3.9. 


Kobra 

Kobra  combat 

conducting  up 

incapable  for 

to  1 0  terrorist 

more  than  6 

attacks  daily. 

mos. 

Figure  3.9  Effects-Tree  Diagram  Example — Step  1 

Figure  3.10  depicts  the  indicators  that  immediately  come  to  mind  to  assess  the 
state  of  Kobra.  Assessors  often  stop  at  this  point  when  trying  to  determine  success 
indicators  for  military  actions.  However,  tracking  only  one  indicator  to  determine  the 
effect  of  multiple  military  actions  has  two  limitations:  1)  the  indicator  may  not  be 
measurable  in  practice;  and  2)  the  indicator  may  change  based  on  other  factors 
independent  of  friendly  actions.  The  first  limitation  could  result  in  assessors  having  a 
non-observable  indicator  on  which  to  base  the  assessment  of  an  effect.  The  second 
limitation  could  result  in  assessors  making  incorrect  assessment  based  on  what  appears  to 
be  a  direct  causation.  In  this  example,  the  number  of  Kobra  attacks  will  likely  be  affected 
by  friendly  actions,  but  Kobra  attacks  also  may  be  affected  by  other  factors  internal  to  the 
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Kobra  organization.  Assessors  want  to  know  the  impact  of  the  synergy  of  friendly 
actions  on  the  system  and  view  how  the  system  changes  as  a  result  of  those  actions. 
Developing  multiple  indicators  will  aid  this  goal. 


Figure  3.10  Effects-Tree  Diagram  Example — Step  2 


Step  3  (Figure  3.11)  begins  to  look  at  the  impact  of  each  friendly  action.  In  this 
example,  the  planners  have  decided  on  a  four  actions  against  four  separate  subsystems  of 
the  Kobra  terrorist  organization:  neutralize  command  and  control  (C2)  leadership; 
destroy  Kobra  sanctuary;  destroy  Kobra  military  equipment;  and  influence  Kobra  fielded 
forces  to  capitulate. 


Neutralize  Destroy  Influence 


C2  Destroy  Military  Fielded  Forces 

Leadership  Sanctuary  Equipment  to  Capitulate 


Figure  3.11  Effects-Tree  Diagram  Example — Step  3 
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Figure  3.12  shows  the  initial  branching  of  the  diagram  with  a  branch  for  each 
planned  action  against  the  system.  These  branches  represent  the  indirect  effects  resulting 
from  the  achievement  of  the  desired  effect  given  the  action  is  successful  in  isolation  from 
the  other  actions.  Figure  3.13  depicts  the  completion  of  the  effects  branch  with  the 
“Neutralize  C2  Leadership”  branch,  showing  the  anticipated  indirect  effects  and  their 
associated  measurable  indicators.  The  full  effects-tree  diagram  for  this  example  is  given 
in  Appendix  A.  The  list  of  unique  indicators  developed  from  this  diagram  are  given  in 
Table  3.1. 


Kobra 

conducting  up 
to  1 0  terrorist 
attacks  daily. 


Neutralize 

C2  Destroy 

Leadership  Sanctuary 

AA 


Destroy  Influence 

Military  Fielded  Forces 
Equipment  to  Capitulate 


Increase 
in  number 
of  attacks 


Effect  |  Destroy 
^anctuar^^^. 


Effect  |  Destroy  Mil 
Equipment 


Kobra  combat 
incapable  for 
more  than  6 


Effect  |  Forces 
Capitulate 


No  attacks  _ 


Effect  |  Neutralize 
C2  Leadership 


Figure  3.12  Effects-Tree  Example — Step  4 
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Kobra 

conducting  up 
to  1 0  terrorist 
attacks  daily. 


Neutralize 

C2  Destroy 

Leadership  Sanctuary 


Destroy  Influence 

Military  Fielded  Forces 
Equipment  to  Capitulate 


X  \  X  X 


Effect  |  Destroy 
^Sanctuar^^^ 


Effect  |  Destroy  Mil 
^Equipment^^^ 


Effect  |  Forces 
Capitulate 


Kobra  combat 
incapable  for 
more  than  6 
mos. 


Increase 
in  number 
of  attacks 


Attacks/day 

decrease 


Effect  |  Neutralize 
C2  Leadership 


Decrease 
in  Ops 


Decrease  in  press 
releases  ^ 


New  leaders 
emerge 


New  names/faces 
in  press  releases 


Factions 

emerge 


New  organizations 

Claiming 

responsibility 


Uncoordinated 

Ops 


Increase  in 
Number  of 
isolated  ops 


Out  dated 

audio/video 

releases 


No  attacks  _ 


Figure  3.13  Effects-Tree  Example — Step  5 


Table  3.1  Indicators  Generated  from  Effects-Tree  Diagram  Example 


Neutralize  C2 
Leadership 

Destroy  Sanctuary 

Destroy  Military 
Equipment 

Influence  Fielded 
Forces  to 
Capitulate 

Attacks/day 

decrease 

Attacks/day  decrease 

Increased  use  of 
improvised  explosives 

Attacks/day  decrease 

Decrease  in  press 
releases 

Decrease  in  press 
releases 

Decrease  in  attacks 
aimed  at  high 
casualties 

Decrease  in  press  release 

New  name/faces  in 
press 

Increased  traffic  at 
borders 

Increase  in  attacks 
aimed  at  infrastructure 

Increased  attacks  on  high 
visibility  targets 

New  organizations 

claiming 

responsibility 

Increased  rhetoric  in 
bordering  countries 

Increased  traffic  at 
borders  (smuggling) 

Increased  attacks  on  high 
casualty  targets 

Increase  in  number 
of  isolated  ops 

Increased  use  of 
alternate  comm 

Increased  attacks  on 
coalition  facilities 

Increased  demonstrations 

Dated  audio/video 
releases 

Increased  thefts  from 
security  forces 

Increased  traffic  to 
training  camps 

Small  scale  attacks 

Increase  web  postings  of 
suicide  bombers 

Increased  threats  and 
press  releases 
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In  Phase  II,  assessors  make  three  crucial  assumptions:  1)  the  action  under 
consideration  is  100-percent  successful  in  creating  the  desired  effect;  2)  the  action  is  the 
only  action  that  has  taken  place  to  achieve  the  desired  effect;  3)  the  enemy  will  continue 
to  fight  by  any  means  necessary.  The  first  two  assumptions  are  made  in  order  to  generate 
potential  indirect  effects  and  for  that  purpose  alone.  They  do  not  in  any  way  represent 
the  intentions  of  the  planners  that  each  action  will  be  the  sole  means  to  achieve  the 
desired  effect.  By  making  these  assumptions  with  each  effect-tree  diagram,  assessors  are 
able  to  generate  a  list  of  many  possible  enemy  reactions  that  might  result  from  each  one 
of  the  friendly  actions.  The  assumptions  alleviate  the  problem  of  detennining  the 
reactions  from  all  possible  actions  simultaneously.  Viewing  the  enemy  through  these 
assumptions  allows  planners  to  conceive  possible  indirect  effects  one  at  a  time. 

When  the  diagram  is  complete,  the  assessors  will  have  a  list  of  measurable 
indicators.  These  indicators  taken  together  represent  how  the  enemy  might  react  to  all  of 
the  friendly  actions  executed  in  EBE.  Before  execution,  the  assessors  know  with 
certainty  neither  how  successful  the  friendly  actions  will  be  nor  how  the  enemy  will  react 
to  said  actions.  The  list  of  indicators  then  represents  what  might  happen  given  any 
combination  of  friendly  actions  and  enemy  reactions.  Though  developed  under  strict 
assumptions,  each  indicator  combined  with  other  indicators  provides  a  range  of  possible 
enemy  system  reactions.  In  the  next  phase,  this  range  will  be  described  qualitatively  by  a 
series  of  ordinal  system  states.  The  set  of  indicators  will  then  be  partitioned  according  to 
these  states. 
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3.3.3  Phase  III:  Defining  System  States  by  Partitioning  the  Indicators. 


In  order  to  track  progress  towards  the  achievement  of  a  desired  effect,  the  EBA 
process  must  track  the  state  of  the  systems  of  interest  as  the  systems’  state  changes. 
Knowing  the  state  of  the  system  will  give  insight  into  the  effectiveness  of  the  actions 
taken.  The  indicators  developed  in  Phase  II  will  be  used  for  this  purpose. 

In  Phase  I,  the  initial  and  desired  end  states  of  the  system  are  defined.  However, 
other  possible  intermediate  states  are  not  always  defined  a  priori  as  the  initial  and  desired 
states  are  defined.  When  this  is  the  case,  the  intermediate  states  can  be  defined  by 
partitioning  the  indicators.  The  assessors  can  bin  together  indicators  that  represent 
similar  levels  of  consequences  under  the  same  label  to  fonn  an  ordinal  scale  of  system 
states.  The  intermediate  states  are  essentially  pseudo  states.  These  pseudo  states,  while 
not  precisely  states  of  the  system,  are  a  collection  of  indicators  by  which  assessors  can 
measure  progress  towards  the  desired  end  state. 

Phase  III  describes  two  ways  to  partition  the  set  of  indicators  into  states.  Both 
methods  place  each  indicator  into  a  predefined  state  of  ordinal  consequences.  The  first 
ordinal  scale  is  the  familiar  stoplight-chart  assessment  states:  red,  amber,  and  green.  The 
second  scale  is  more  detailed,  consisting  of  five  states.  This  scale  is  similar  to  the 
consequence  scale  presented  by  Haimes  in  the  Risk  Filtering  and  Ranking  Methodology 
for  risk  assessment  (reference  Section  2.13).  In  the  stoplight  chart  method  for 
partitioning  the  indicators,  each  color  of  the  stoplight — red,  amber,  and  green — 
represents  a  state  of  the  system  (Figure  3.14).  Red  is  the  least  desired  state  as  defined  by 
the  commander,  amber  is  a  moderate  or  acceptable  level  given  the  progress  of  operations, 
and  green  is  the  desired  end  state.  Planners  and  assessors  examine  each  indicator 
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developed  in  Phase  II  and  determine  which  system  state  the  indicator  best  represents. 
The  mechanism  for  partitioning  the  indicators  is  the  question  “if  this  indicator  occurs, 


what  is  the  most  likely  state  of  the  enemy  system?” 


System  is  still  in  the 
initial  state  or  has 
undergone  a  minor 
change. 


System  has  undergone 
a  significant  change. 


System  is  in  the 
desired  end  state. 


Figure  3.14  Stoplight  Chart  System  States 

The  five-state  consequence  scale  (5S),  pictured  in  Figure  3.15,  is  similar  to  the 
stoplight  scale  but  has  more  intermediate  states.  More  states  allows  assessors  to  view 
progress  over  time  more  easily  and  gives  a  greater  fidelity  to  the  overall  EBA  process.  In 
the  first  state  of  5S  after  actions  have  been  taken,  the  system  has  undergone  little  or  no 
change  and  is  effectively  still  in  s° .  In  the  second  state,  the  system  has  undergone  some 
recognizable  change  but  the  function  of  the  system  is  the  same.  In  the  third  state,  the 
system  has  undergone  a  change  and  the  system  function  has  been  altered  to  a  noticeable 
degree.  In  the  fourth  state,  the  system  function  has  changed  significantly,  but  the  system 
is  not  yet  in  the  desired  state.  Finally,  the  fifth  state  is  the  desired  end  state  of  the  system. 
Figure  3.15  summarizes  example  state  descriptions  for  changes  in  capability  and  changes 
in  behavior. 
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When  Phase  III  is  complete,  every  indicator  has  been  assigned  to  one  of  the 
defined  states.  For  a  given  effect,  all  indicators  developed  in  Phase  II  must  be  attached  to 
a  state  of  the  system,  and  every  state  must  have  at  least  one  indicator.  If  either  condition 
is  not  met,  Phase  II  may  need  to  be  revisited  to  generate  more  indicators,  or  the  state 
descriptions  need  to  be  redefined  to  lessen  the  number  of  states.  The  number  of  states 
may  vary;  this  thesis  only  presents  three-state  and  five-state  models  as  examples.  Two- 
state  or  four-state  models  may  better  represent  the  enemy  system  state  transitions  that 
result  from  friendly  actions.  Assessors  should  readjust  both  the  set  of  indicators  and  the 
ordinal  scales  of  system  states  as  necessary.  Remember,  the  goal  of  Phase  III  is  to 
specify  a  set  of  states  that  the  system  will  likely  transition  into  as  result  of  friendly 
actions,  and  to  conduct  an  appropriate  partitioning  of  the  set  of  indicators  to  describe  the 
conditions  of  each  state.  As  an  example,  consider  the  Exstan  indicators.  Table  3.2  shows 
a  partition  of  the  indicators  listed  previously  in  Table  3.1. 


Capability 

S°:  Minor  or  no  change  in 
capability 

S1:  Some  change  in  capability; 
no  effect  on  mission 

S2:  Loss  of  capability  with  some 
effect  on  mission 

S3:  Loss  of  mission 

S4:  Complete  loss  of  system 
capability  or  desired  capability 
attained 


Behavior 

S°:  Minor  or  no  change  in 
behavior 

S1:  Some  change  in  behavior 

S2:  Noticeable  changes  in 
behavior;  other  behaviors  evident 

S3:  Original  behavior  is  ceased 

S4:  Desired  behavior 


Figure  3.15  Five-State  Transition  Model  for  Capabilities  and  Behaviors 
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Table  3.2  Example  of  Indicator  Partitioning 


m 

c 

°  Kobra 

s° 

s1 

s9 

s3 

se 

State 

Description 

Little  or  no  change 
in  warfare  methods 

Some  change  in 
methods  with  little  or 
no  effect  on  overall 
combat  capability 

Significant  change 
in  methods  with 
some  effect  on 
combat  capability 

Complete  change  in 
method 

Complete  loss  of 
combat  capability 

Increased  use  of 
IEDs 

Decrease  in  press 
releases 

Increased  threats 
and  press  releases 

Increased 

demonstrations 

No  attacks. 

New  names  and 
faces  in  leadership 
positions  in  press 

Decrease  in  attacks 
aimed  at  high 
casualties 

Decrease  in  number 
of  attacks  per  day 

Increased  rhetoric 
in  bordering 
countries 

Dated 

audio/video 

releases 

Increased  use  of 
alternate  comm 

Increased  attacks  on 
high  visibility  targets 

Increased  attacks  on 
infrastructure 

Increased  thefts 
from  security 
forces 

Indicators 

Increased  attacks 
on  coalition 
facilities 

Increased  number  of 

uncoordinated/isolated 

ops 

Small  scale  attacks 

Increased  attacks 
on  high-casualty 
targets 

New  organizations 
claiming 
responsibility  for 
attacks 

Increased  web 
postings  of  suicide 
bombers 

Increased  traffic  to 
training  camps 

Increased 

smuggling/traffic  at 
borders 

Increased  traffic  at  the 
borders 

3.3.4  Phase  IV:  Assessing  Friendly  Actions  Taken  against  Enemy  Systems. 

Phase  IV  describes  the  basic  elements  of  assessing  the  completion  of  coalition 
actions  prescribed  by  the  COAs  defined  in  EBP.  Tracking  actions  is  nothing  new.  The 
process  described  in  this  thesis  is  very  similar  to  the  process  used  in  OIF  to  track  the 
completion  of  CFACC  tasks.  Essentially,  assessors  assign  Measures  of  Performance 
(MOPs)  to  the  actions  and  track  the  MOPs. 

For  each  action  of  an  effect,  planners  have  a  set  of  action-node  pairings.  The  goal 
of  the  actions  assessment  is  to  determine  if  the  action  has  been  completed  to  the  level 
required  to  produce  the  effect  on  the  node.  In  order  to  make  this  determination,  assessors 
examine  the  nodes.  For  each  action-node  pairing  assessors  develop  MOPs. 

Combat  actions  often  take  on  similar  forms,  and  their  associated  MOPs  are  also 
similar.  Since  MOPs  have  been  used  in  the  past,  Phase  IV  can  draw  from  the  format  of 
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past  MOPs.  For  example,  actions  of  the  type  “destroy  enemy  nodes”  can  be  measured 
with  an  MOP,  “percent  of  enemy  nodes  destroyed.”  This  MOP  can  then  be  assessed  via  a 
physical  assessment.  In  order  to  determine  that  enough  enemy  nodes  have  been 
destroyed,  certain  parameters  are  required.  Assessors  must  know  the  initial  number  of 
existing  nodes  and  the  number  of  nodes  required  to  achieve  the  effect.  The  initial  number 
of  existing  nodes  is  called  the  baseline.  The  number  of  nodes  required  to  achieve  the 
effect  is  called  the  goal  level  or  target  level.  It  is  possible,  and  often  is  the  case,  that  100 
percent  of  the  enemy  nodes  need  be  destroyed  in  order  to  create  the  effect.  Sometimes, 
however,  not  all  of  the  nodes  need  be  destroyed.  Whatever  the  level  of  nodes  needing  to 
be  destroyed,  the  assessors  need  to  declare  it. 

Thoele  et  al  offer  a  short  list  of  predefined  MOPs  based  on  coalition  tasks  in  OIF 
(reference  section  2.5).  Table  3.3  summarizes  example  actions  and  MOPs  based  on  the 
OIF  assessment  model.  Once  the  MOPs  have  been  defined,  just  as  with  the  indicators  in 
Phase  II,  the  MOPs  should  be  coordinated  with  the  ISR  division  to  acquire  data  for  the 
baselines  and  target  values  as  well  as  to  develop  a  collection  plan  for  tracking  MOPs 
when  combat  operations  commence. 


Table  3.3  Example  MOPs  for  Friendly  Actions  (Thoele,  DiSebastian,  and  Garcia,  2004) 


Coalition  Action 

Example  MOP 

Degrade/Disrupt/Destroy/Neutralize 
the  enemy  nodes 

Percentage  of  enemy  nodes 
degraded,  disrupted,  destroyed,  or 
neutralized 

Detect  and  Identify  specific  enemy 
capabilities  (GPS  jamming,  etc.) 

Number  of  occurrences  of  enemy 
capability  detected  and  identified 

Maintain  24-hour  coverage  or 
specific  coalition  capability 

Percentage  of  coverage  or 
capability  achieved  over  24-hour 
period 
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In  order  to  aid  the  planners,  tracking  the  MOPs  should  include  a  time  dimension 
in  order  to  view  trends.  That  is,  the  level  of  the  MOP  should  be  plotted  against  a  time 
axis  for  each  day  of  the  campaign  (Figure  3.16).  The  actions  may  also  be  assessed 
collectively.  To  determine  the  accomplishment  of  a  set  of  actions  for  an  effect,  a  linear 
additive  value  function  may  be  used  wherein  each  action  MOP  is  given  a  weight 
w  e  [0, 1]  as  it  applies  to  the  effect  and  ^  wt ■=  1  .  Then  the  completion  of  all  actions  for 

i 

an  effect  may  be  given  by  a  value  function 


where  w,  is  the  weight  of  the  action  and  c,  is  the  level  of  completion  of  the  action, 


c  e  [0, 1] .  The  value  v  can  be  plotted  over  time  as  well. 


1 


0.1 


0 


D+1  D+2  D+3  D+4  D+5  D+6  D+7  D+8  D+9  D+10  D+11  D+12 

Campaign  Timeline 


Figure  3.16  Actions  Completion 
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Assessing  coalition  actions  is  a  necessary  piece  of  EBA,  answering  the  question 
“How  well  is  the  plan  being  executed?”  For  each  action,  assessors  can  define  MOPs 
based  on  previous  assessment  efforts  or  develop  new  MOPs  appropriate  for  the  COA. 
The  MOPs  can  be  tracked  individually  or  aggregated  as  they  support  an  effect.  To  aid 
planners,  the  level  of  completion  of  the  actions  should  be  plotted  in  a  graph  against  time 
to  view  developing  trends.  The  actions  assessment  will  be  used  along  with  the  effects 
assessment  (Phase  V)  to  build  a  campaign  assessment  chart  in  Phase  VII. 

3.3.5  Phase  V:  Assessing  Effects  based  on  Indicators  of  Enemy  Reactions. 

As  aforementioned,  an  effect  is  a  change  in  state  A.v  of  an  enemy  system. 
Friendly  actions  are  synchronized  to  produce  the  desired  effect 

As  =  s°  ^se 

If  the  system  state  transitions  are  defined  as  s°  — >  s1  — >  s2  — >  V  where  s°  is  the  initial 
state  and  s3  is  the  desired  state,  the  desired  effect  is  A.v  =  ,v°  — >  s3 .  However,  any  state 
change  such  as  sn  — >  s1  can  also  be  viewed  as  an  effect  on  the  system.  By  monitoring 
these  changes  in  system  states,  assessors  aim  to  view  progress  towards  achieving  the 
desired  effects.  Therefore,  assessors  must  determine  the  system  states  to  determine 
which  Ay  has  occurred. 

In  order  to  determine  what  effects  friendly  actions  have  had  on  the  system  of 
interest,  assessors  need  to  determine  the  state  of  the  system.  The  state  of  the  system  can 
be  described  by  the  set  of  indicators  defined  in  earlier  phases  of  the  EBA  process.  More 
specifically,  this  relationship  is  described  by  the  function  g  :  /  — »  S  such  that 
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sm  =g,„0'i>4>-4) 


where  and,  Im  =  {ix,i2...,ir }  is  a  set  of  indicators  i,  i  e  {0,1}  .  When  a  predefined  number 
of  indicators  exists  for  a  state  sm  e  S,  then  the  system  of  interest  n^j  is  said  to  be  in  s’" . 
Essentially,  g(-)  is  based  on  an  assessment  of  whether  or  not  enough  evidence  exists  (via 
the  indicators)  to  declare  a  system  in  a  particular  state.  To  aid  assessors,  this  thesis 
advocates  the  use  of  the  40-percent  rule  inspired  by  Powell  (reference  section  2.10). 

The  forty-percent  rule  simply  states  that  if  assessors  have  evidence  of  at  least  40 
percent  of  the  indicators  of  a  given  state,  then  it  is  reasonable  to  conclude  that  the  system 
is  in  that  state.  In  this  method,  the  weight  of  each  indicator  is  normalized  with  the  other 
indicators  within  that  state,  so  that  each  indicator  provides  equal  weight  of  evidence  for 
the  state.  Therefore,  the  weight  of  an  indicator  is  a  function  of  the  number  of  indicators 
in  that  state.  That  is  to  say,  if  state  A  has  more  indicators  assigned  to  it  than  state  B, 
assessors  will  need  to  witness  more  of  the  indicators  of  state  A  to  declare  the  system  in 
state  A  than  to  declare  it  in  state  B.  This  condition  agrees  with  the  logic  used  to  create 
the  indicators.  The  assumption  is  that  all  indicators  are  equally  likely  to  occur.  Given  the 
coalition  actions  taken,  it  is  not  known  with  certainty  how  the  system  will  react.  Thus  the 
more  indicators  showing  evidence  of  a  system  being  in  state  s'"  the  greater  the  chance 
that  the  system  is  in  that  state. 

Using  the  40-percent  rule,  assessors  can  reasonably  conclude  that  the  system 
resides  in  the  highest  state  for  which  greater  than  or  equal  to  40  percent  of  its  indicators 
have  been  observed.  (In  this  thesis,  higher  denotes  a  state  that  is  closer  to  the  desired 
state.)  Problems,  however,  arise  when  evidence  exists  for  indicators  in  multiple  states 
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and  no  state  has  evidence  of  40  percent  of  its  indicators,  or  when  multiple  states  have 
evidence  for  greater  than  40  percent  of  their  indicators.  How  then  do  assessors  decide 
which  state  the  system  is  in?  The  question  rephrased  asks,  “given  the  evidence  of  system 
indicators,  what  is  the  likelihood  that  the  system  is  in  a  particular  state?”  The  following 
guidelines  provide  some  answers  to  this  question: 

1)  The  system  has  the  highest  likelihood  of  being  in  the  state  with  the  highest 
percentage  of  its  indicators  observed. 

2)  If  there  is  a  tie  in  the  percentage  of  observed  indicators  for  multiple  states, 
then  it  is  more  likely  that  the  system  is  in  the  higher  state.  This  conclusion  is 
based  on  the  following  assumptions:  given  that  the  system  is  actually  in  the 
lower  state,  the  likelihood  of  seeing  the  higher-state  indicators  is  quite  low. 
Conversely,  given  the  system  as  actually  in  the  higher  state,  the  likelihood  of 
seeing  the  lower-state  indicators  is  high.  This  assumption  is  based  upon  the 
fact  that  the  coalition  actions  are  designed  to  achieve  the  higher  system  states. 
Assessors  expect  the  actions  to  achieve  the  higher  states.  However,  caution  is 
warranted  in  this  situation.  While  the  presence  of  an  indicator  in  the  higher 
state  is  evidence  that  the  system  may  be  in  that  state,  without  additional 
evidence  it  may  be  prudent  to  proceed  as  though  the  system  is  in  the  lower 
state  until  more  evidence  is  available. 

3)  If  multiple  states  have  evidence  of  more  than  40  percent  of  the  indicators,  then 
it  is  more  likely  that  the  system  is  in  the  higher  state.  (This  is  based  on  the 
same  assumptions  as  number  2.) 
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As  in  all  assessment  efforts,  however,  the  point  of  the  guidelines  are  to  provide  insight. 
Subject-matter  experts  (SME) — i.e.,  system  experts,  intelligence  operators,  leadership 
with  other  sources  of  infonnation — and  operational  knowledge  gained  from  experience 
can  trump  the  state  assessment  provided  by  the  guidelines.  Likewise,  in  practice,  actual 
enemy  reactions  may  not  correspond  directly  to  the  predefined  indicators.  When  this 
occurs,  assessors  can  use  Smith’s  questions  (reference  Section  2.7.5)  as  catalysts  for 
assessing  the  state  of  an  enemy  system.  Whether  the  assessment  is  made  via  the 
indicators,  SME  opinion,  or  ad  hoc  based  on  an  unanticipated  enemy  reaction,  the  goal  of 
Phase  V  is  to  determine  the  state  of  the  systems  of  interest  in  terms  of  the  desired  effects. 
These  state  assessments  will  provide  inputs  to  the  objective  progress  function  in  Phase 
VI. 


3.3.6  Phase  VI:  Assessing  the  Progress  of  the  Objective  based  on  the  State  of 
the  Enemy  Systems. 

In  order  to  assess  the  progress  of  objective  k,  assessors  need  to  define  the  progress 
function  Ok  for  the  objective.  Phase  VI  describes  how  to  define  this  function.  First,  the 
no-attainment  and  full-attainment  regions  are  defined  by  two  sets  of  state  vectors.  These 
regions  are  the  ones  that  map  to  p  =  0  and  p  =  1  respectively.  Next,  the  remaining  state 
vectors  are  binned  together  into  a  middle  region  for  which  the  /^-values  will  be  later 
assigned.  Then,  the  projected  progress  curve  is  defined  based  on  the  time-series  actions 
developed  in  EBP  and  the  anticipated  resulting  state  vectors.  The  anticipated  state 
vectors  are  assigned  /^-values  based  on  planners’  input.  Finally,  as  the  campaign  is 
executed  and  assessments  are  made  regarding  the  states  of  the  systems,  those  states  are 


assigned  approximate  /^-values  via  pair-wise  comparisons  with  the  existing  state  vectors, 
thus  completing  the  progress  function. 

The  no-attainment  and  full-attainment  regions  correspond  to  the  red  and  green 
states  of  a  stoplight  chart.  Using  the  stoplight-chart  nomenclature  works  particularly  well 
in  an  operational  setting  as  commanders  are  used  to  seeing  operational  objectives 
described  in  terms  of  red,  amber,  and  green  (reference  Section  2.5).  To  map  state  vectors 
to  the  green  and  red  regions,  assessors  need  only  choose  those  vectors  which  describe  full 
attainment  of  the  objective  and  those  that  describe  no  attainment  of  the  objective.  There 
will  always  be  at  least  one  vector  in  each  of  these  regions;  the  vector  (jj°,  j®,— in  the 

red  region,  and  the  vector  (s[,se2,...,sen)  in  the  green  region.  The  red  and  green  regions 

are  then  assigned  the  /^-values  0  and  1  respectively.  Any  vectors  not  mapped  to  red  and 
green  regions  fall  into  the  amber  region  by  default. 

Assessors  next  build  a  progress  curve  based  on  the  planned  time-phase  actions 
and  anticipated  enemy  system  states  resulting  from  those  actions  as  described  in  EBP. 

The  end  points  of  the  progress  curve  can  be  derived  directly  from  the  definition  of  the 
desired  effect  from  Phase  I.  At  the  start  of  the  campaign  (D+0),  all  systems  are  in  their 
initial  states.  Likewise,  the  desired  end  state  of  a  system  is  planned  to  be  achieved  by  the 
start  date  from  the  desired  effect  definition.  Figure  3.17  illustrates  the  following 
example:  Let  objective  k  have  two  systems  of  interest,  each  with  five  possible  states 
(including  the  initial  state).  In  this  example,  the  desired  end  state  for  both  systems  is  to 
start  on  D+5.  The  planners  anticipate  that  after  the  first  day  of  the  campaign,  both  of  the 
enemy  systems  will  have  transitioned  into  state  two.  Similarly,  after  three  days  of  the 
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campaign,  both  systems  will  have  transitioned  into  state  three.  Therefore,  the  planners 
have  anticipated  that  the  vector  states  will  transition  as  follows: 

(0, 0)  — » (2, 2)  — » (3, 3)  — »  (4, 4) 

The  question  the  assessors  must  then  ask  the  planners  is  “how  much  of  the  objective  will 
be  attained  at  each  of  the  anticipated  state  vectors?”  In  this  example,  the  planners  think 
approximately  30  percent  of  the  objective  will  be  attained  after  the  first  day  (state  vector 
(2,2)).  Likewise,  approximately  70  percent  of  the  objective  will  be  attained  after  the  third 
day  (state  vector  (3,3)).  The  state  vectors  can  then  plotted  on  a  timeline  against  the 
percentage  of  the  objective  attained.  Connecting  the  points  then  results  in  an  a  progress 
curve  for  which  assessors  can  compare  the  assessed  progress  as  EBE  commences  during 
Phase  VII.  Figure  3.17  helps  illustrate  this  process. 


100% 


Figure  3.17  Projected  Progress  Function 
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The  assigned  levels  of  attainment  for  the  anticipated  state  vectors  begin  the 
progress  function.  Figure  3.18  illustrates  the  progress  function  based  on  these  initial 
mappings. 


100% 


State  Vectors 


Figure  3.18  Progress  Function  After  Initial  Mappings  of  State  Vectors 

At  this  point,  only  a  fraction  of  the  possible  state  vectors  have  been  mapped  to 
progress  values:  the  no-attainment  vectors  (p  =  0),  the  full  attainment  vectors  (p  =  1), 
and  the  anticipated  state  vectors  that  the  planners  have  mapped  to  approximate  /> values. 
The  final  item  in  this  phase  is  to  map  the  remaining  state  vectors  to  p- values.  Put  another 
way,  for  each  state  vector  sk ,  assessors  assign  sk  to  a  value  / (sA.)  on  the  “percent  of 
objective  attainment”  line. 

Mapping  all  possible  state  vectors,  however,  may  be  intractable.  Even  for  a  small 
number  of  systems  and  the  associated  system  states,  for  each  system  q,  the  number  of 
possible  state  vectors  can  be  quite  large.  In  fact,  for  any  number  tp  systems  and  number 
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nk 

of  states  of  the  system  mq,  the  number  of  st  vectors  is  equal  to  ]” [  mq  .  From  the  earlier 

g= i 

example  in  Section  3.2.1,  with  two  systems  having  six  states  each  (including  the  initial 
state),  rik  =  2  and  mq  =  6  for  all  q,  which  yields  6x6  =  36  different  state  vectors  that  need 
to  be  considered.  As  nk  and  mq  increase,  the  number  of  state  vectors  could  quickly 
explode. 

This  state  vector  explosion  problem  is  not  intractable.  With  an  initial  progress 
function  defined,  all  that  is  required  is  to  map  the  state  vectors  that  occur  as  a  result  of 
friendly  actions  during  EBE  when  they  occur.  That  is,  as  assessors  determine  the  states 
of  the  systems  of  interest,  they  need  only  determine  in  which  interval  of  the  progress 
function  the  vector  belongs  and  approximate  the  value  associated  with  it.  Again,  the 
purpose  of  the  progress  function  is  to  show  progress  towards  the  attainment  of  the 
objective.  Exact  measurements  are  not  as  important  as  ordinal  relationships;  assessors 
need  only  to  know  approximate  relationships  between  the  state  vectors  to  determine  if  the 
commander  is  getting  closer  to  objective  attainment  or  further  away  from  it.  Another 
example  helps  illustrate  this  point.  Let  the  state  vector  (3,1)  be  the  vector  assessed  after 
the  first  day  of  the  campaign.  Then  assessors  must  map  (3,1)  to  a  /7-value.  Figure  3.19 
shows  that  the  planners  have  determined  that  (3,1)  yields  attainment  of  the  objective 
somewhere  between  (2,2)  and  (3,3),  closer  to  (3,3)  than  to  (2,2).  Furthermore  the 
planners  detennine  the  level  objective  attainment  to  be  about  55%,  or  a  /7-value  of  0.55. 
The  assessors  can  then  add  to  the  categorical  progress  function  as  shown  in  Figure  3.20. 
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Figure  3.19  Mapping  new  state  vectors  individually 
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Figure  3.20  Discrete  progress  function  after  first  day  of  campaign. 


3.3.7  Phase  VII:  Campaign  Assessment. 

Phase  VII  combines  the  results  of  Phases  IV,  V,  and  VI  with  the  time  dimension. 
Phase  VII  allows  planners  and  assessors  to  view  the  objective  attainment  and  actions 
accomplishment  over  time  to  get  an  overall  picture  of  the  evolution  of  the  entire 
campaign.  The  planners  chose  the  actions  in  order  to  achieve  the  desired  effects  and 
attain  the  objectives.  In  this  phase,  assessors  can  view  whether  the  enemy  is  responding 
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as  expected  to  the  friendly  actions  as  well  as  view  whether  or  not  the  effects  are  being 
achieved  in  the  planned  timeframe. 

If  the  effects  have  been  achieved  yet  the  actions  have  not  been  accomplished,  or  if 
the  actions  have  been  accomplished  and  the  effects  have  not  yet  been  achieved,  then 
important  questions  need  to  be  investigated.  In  either  case,  the  relationship  between  the 
actions  and  effects  is  in  question.  The  SoSA  model  may  need  to  be  reconsidered,  as  well 
as  the  knowledge  of  the  enemy,  the  indicators,  and  actions  chosen  by  the  planners. 

Figure  3.21  illustrates  an  example  view  of  objectives  versus  actions  over  time. 

Plotting  the  levels  of  action  completion  versus  the  objective  attainment  can 
provide  insight  by  describing  the  relationships  between  the  actions  and  effects.  As  in 
Figure  3.22,  a  general  fonn  of  a  functional  relationship  may  be  able  to  be  determined.  If 
the  plot  looks  like  line  A  then  the  CFACC  is  experiencing  diminishing  returns  on  the 
actions  taken.  Line  B  represents  a  linear  relationship  between  the  CF ACC’s  actions  and 
the  effects.  Line  C  shows  the  CFACC  experiencing  increasing  returns  on  his  actions. 
Graphs  such  as  these  can  help  planners  adjust  weights  of  effort  to  support  lagging  effects 
and  objectives.  Regression  analysis  could  also  be  used  to  look  at  the  contribution  of  the 
individual  actions  with  regards  to  the  effects. 
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Figure  3.21  Effects  and  Actions  over  Time 
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Figure  3.22  Objectives  vs  Actions  Graph 


Defining  a  planned  progress  curve  in  Phase  VI  allows  assessors  to  compare  the 
actual  progress  over  time  as  EBE  commences.  On  a  graph  such  as  Figure  3.23,  planners 
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and  commanders  can  see  when  progress  shortfalls  have  taken  place  and  adjust  future 
plans  accordingly.  Figure  3.23  builds  on  the  example  from  Section  3.3.6.  In  this 
example  the  friendly  actions  taken  have  produced  state  vector  (3,1)  on  campaign  day 
D+l,  and  state  vector  (3,2)  on  D+3  and  D+5.  As  these  state  vectors  occur,  planners  rate 
them  according  to  their  approximate  attainment  of  the  objective — (3,1)  rates  55  percent 
attainment  and  (3,2)  rates  60  percent  attainment.  When  compared  to  the  planned 
progress  curve,  planners  and  assessors  can  see  that  the  progress  was  ahead  of  schedule  on 
D+l  but  then  fell  behind  schedule  for  the  remaining  assessment  days.  Assessors  can  then 
investigate  the  actions  accomplished  on  those  days  along  with  the  effects  achieved  to 
determine  the  cause  of  the  progress  shortfall.  Planners  can  then  adjust  the  plan  to  return 
to  schedule,  adjust  the  schedule,  or  determine  another  course  of  action  as  needed. 


100% 


Figure  3.23  Planned  Progress  vs.  Actual  Progress 
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In  summary,  campaign  assessment  combines  actions  assessment,  effects 
assessment,  and  objective  assessment  in  the  temporal  domain  to  determine  how  the 
effects  respond  to  the  actions  taken.  Plotting  the  objectives  and  actions  together  can 
reveal  this  relationship.  Likewise,  plotting  the  planned  progress  curves  versus  the  actual 
progress  curves  can  reveal  important  insight  to  planners  and  commanders. 

3.4  Conclusion 

The  goal  of  EB A  is  to  provide  insight  to  commanders  and  planners  about 
coalition  actions,  enemy  reactions,  achievement  of  effects,  and  attainment  of  objectives. 
The  mathematical  construct  described  in  the  first  part  of  the  chapter  defines  the  progress 
function,  states,  state  vectors,  and  regions  in  order  to  provide  the  foundation  for 
implementation  of  the  EBA  methodology.  In  order  to  be  useful  in  an  operational 
environment,  assessors  must  work  with  planners  from  the  beginning  of  EBP  to  create  an 
assessable  plan.  Using  the  tools  and  methods  described  in  the  second  part  of  the  chapter, 
assessors  and  planners  can  develop  a  set  of  indicators  to  determine  the  states  of  systems, 
which  represent  achievement  of  effects.  These  effects  can  then  be  mapped  to  quantities 
via  a  progress  function  defined  with  input  from  the  planners.  Effects  and  actions  can  be 
tracked  over  time  to  view  trends.  Taken  together,  the  actions,  effects,  objectives,  and 
time  provide  commanders  and  planners  with  an  insightful  model  of  the  battlespace. 
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4  Results  and  Analysis 


4.1  Introduction 

Chapter  4  applies  the  methodology  presented  in  Chapter  3  to  two  scenarios.  The 
first  scenario  is  based  on  Operation  DENY  FORCE,  which  describes  a  notional  combat 
operations  air  campaign  set  in  2010.  The  original  scenario  was  developed  for  the  purpose 
of  the  Dynamic  Air  &  Space  Effects-Based  Assessment  (DASEA)  critical  experiment 
(McCrabb,  2005:1).  The  second  scenario  describes  a  notional  stability  operations 
campaign  developed  by  the  author  for  the  purpose  of  this  thesis.  Though  the  focus  of 
Chapter  3  was  on  assessing  the  effects  of  combat  operations,  the  phases  of  the  Effects- 
Based  Assessment  (EBA)  methodology  are  intended  to  be  general  enough  to  be  applied 
to  objectives  and  desired  effects  for  stability  operations  as  well. 

Both  scenarios  are  presented  in  the  same  general  format.  First,  the  general 
background  of  the  scenario  and  the  operational  objectives  are  presented.  Next,  the  first 
three  phases  of  the  EBA  methodology  are  presented  along  with  actions-assessment 
preparation  and  objective  and  campaign-assessment  preparation  for  each  objective. 

Then,  the  Effects-Based  Execution  (EBE)  of  the  campaign  is  briefly  described,  followed 
by  the  last  four  phases  of  the  EBA  methodology. 

4.2  Operation  DENY  FORCE 

US  intelligence  agencies  have  been  monitoring  reports  that  Orangeland  (OL), 
which  the  US  considers  a  “rogue  state”  in  the  region,  has  taken  steps  to  obtain  weapons 
of  mass  destruction  (WMD).  OL  has  long  viewed  the  US  and  its  allies  as  adversaries, 
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while  the  US  allies  in  the  region  look  to  the  US  for  security.  Increasing  evidence  exists 
indicating  that  OL  is  developing  WMD,  especially  research  and  development  into 
chemical,  biological  and  nuclear  weapons  and  the  deployment  of  those  weapons  on 
theater  ballistic  missiles  (TBM).  Ongoing  US  diplomatic  efforts  have  openly  and 
vigorously  protested  OL  activities.  However,  these  efforts  have  been  largely 
unsuccessful  as  OL  denies  all  accusations  involving  WMD.  Meanwhile,  terrorists  in  the 
US  have  executed  multiple  chemical  attacks  in  various  US  cities.  SECDEF  has  ordered 
the  Combined  Forces  Commander  (CFC)  to  begin  contingency  planning  against  OL  with 
the  goal  of  “compel[ling]  Orangeland  to  stop  WMD  or  TBM/CM  development  or 
employment  activities”  (McCrabb,  2005:2).  This  operation  is  called  OPERATION 
DENY  FORCE  (ODF).  The  CFC  has  given  the  Combined  Forces  Air  Component 
Commander  (CFACC)  the  following  operational  objectives: 

Objective  1 :  Gain  and  maintain  air  superiority. 

Objective  2:  Stop  WMD  activities  of  OL  leadership. 

4.2.1  ODF  Objective  1. 

The  first  three  phases  of  the  EBA  methodology,  along  with  actions-assessment 
preparation  and  objective  and  campaign-assessment  preparation  are  presented  for 
Objective  1 :  Gain  and  maintain  air  superiority. 

4.2. 1.1  Phase  I:  Desired  Effect  Definition. 

For  Objective  1  (Gain  and  maintain  air  superiority)  the  planners  define  two 
desired  effects. 
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Desired  Effect  1.1: 


■  System  of  interest :  Airspace  within  Orangeland  Theater  of  Operation  (OLTO). 

■  Initial  State  of  the  System :  OL  possesses  seven  OL  military  air  bases,  conducting 
less  than  25  training  and  combat  sorties  per  month.  The  overall  operational 
capability  of  OL  aviation  is  assessed  as  low,  posing  little  threat  to  coalition  forces. 
Two  major  civilian  airports  have  combined  air  traffic  that  accounts  for  10  to  12 
flights  per  day. 

■  Desired  End  State :  OL  experiences  complete  loss  of  air  sovereignty.  No  OL 
aviation  traffic  without  the  explicit  permission  of  coalition  forces;  “if  it  flies  it 
dies.”  In  addition,  Coalition  forces  have  freedom  of  access  for  follow-on 
persistence  forces. 

■  Start  time :  Effect  to  begin  on  or  before  D+2. 

■  End  time:  Indefinite;  effect  to  be  maintained  as  required  by  CFC. 

Desired  Effect  1.2: 

■  System  of  interest :  OL  Integrated  Air  Defense  Systems  (IADS). 

■  Initial  State  of  the  System :  OL  possesses  approximately  200  Surface  to  Air 
Missiles  (SAM)  and  150  Early  Warning  (EW)  and  Ground  Control  Intercept 
(GCI)  radars.  The  overall  capability  of  the  surface-to-air  threat  is  assessed  as 
medium.  Primary  concerns  are  concentrated  strategic  SAMs  around  OL’s  capital 
and  large  numbers  of  unlocated  tactical  SAMs  throughout  OLTO. 

■  Desired  End  State :  IADS  threat  to  coalition  forces  neutralized. 

■  Start  time :  Effect  to  begin  D+l . 

■  End  time :  Indefinite;  effect  to  be  maintained  as  long  as  required  by  CFC. 
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4.2. 1.2  Phase  II:  Indicator  Development 


The  initial  analysis  for  Desired  Effect  1 .2  is  presented  before  Desired  Effect  1 . 1 
because  Desired  Effect  1 .2  is  expected  to  contribute  to  Desired  Effect  1.1. 

Planners  detennine  four  actions  to  be  taken  in  order  to  achieve  Desired  Effect  1.2: 

■  Destroy  IADS  Command  and  Control  (C2)  facilities. 

■  Destroy  EW/GCI  radars. 

■  Destroy  IADS  communications  facilities. 

■  Destroy  SAM  sites. 

Figure  4.1  depicts  the  effects-tree  diagram  used  to  develop  a  set  of  measurable  indicators 
of  the  desired  effect.  Multiple  actions  will  result  in  the  system  reaction  of  a  decrease  in 
the  number  of  Surface-to-Air  Fires  (SAFIRES).  The  effects-tree  analysis  results  in  six 
unique  indicators  that  will  be  used  to  determine  the  state  of  the  OL  IADS  (shown  in 
Table  4.1). 


Destroy  Destroy 

Destroy  EW/GCI  comm  Destroy 

IADS  C2  radars  facilities  SAM  sites 
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Desired  Effect  1 .2  (IADS  neutralized)  is  the  primary  mechanism  by  which 
planners  aim  to  achieve  Desired  Effect  1 . 1  (Air  Superiority).  Desired  Effect  1 . 1  can  be 
viewed  as  an  indirect  or  secondary  effect  of  achieving  Desired  Effect  1.2.  Figure  4.2 
depicts  the  effects-tree  diagram  resulting  in  the  indicators  given  that  IADS  have  been 
neutralized.  The  effects-tree  diagram  reveals  the  undesired  collateral  effect  that  OL 
reacts  to  the  coalition  air  strikes  by  using  civilian  air  craft  as  WMD.  Since  this  issue  is 
made  known  before  EBE  has  commenced,  the  planners  can  revisit  the  planned  Course  of 
Action  (COA)  and  detennine  if  additional  or  different  actions  are  required  to  mitigate  the 
likelihood  of  OL  using  civilian  aircraft  as  an  offensive  weapon.  For  example,  an  action 
such  as  “destroy  OL  airfield  takeoff  capability”  might  negate  this  collateral  effect.  Figure 
4.2  yields  eight  unique  indicators  to  determine  the  state  of  the  air  space  in  the  OLTO. 
Notice  that  the  indicator  “No  SAFIRES”  is  an  indicator  for  both  Desired  Effect  1 . 1  and 
Desired  Effect  1.2. 


OL  loses  air 
sovereignty/ 
Coalition 
freedom  of 
movement 


No 


Coalition  air 
forces 
uncontested  SAFIRES 
in  OLTO 

- ► 


No  OL  a/c 
airborne 


Figure  4.2  Effects-Tree  Diagram  for  ODF  Desired  Effect  1.1 
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In  general,  indicators  may  overlap  the  desired  effects.  That  is,  the  same  indicator 


may  be  seen  in  more  than  one  desired  effect  as  is  the  case  with  “No  SAFIRES”  for 
Desired  Effects  1.1  and  1.2.  In  practice,  a  single  indicator  may  represent  evidence  for 
system  states  of  multiple  systems.  This  phenomena  is  due  to  the  holistic  understanding 
inherent  in  EBO.  Indirect  effects  may  cascade  throughout  the  enemy  systems,  resulting 
in  similar  effects  on  a  single  system.  This  phenomena  can  aid  assessors  by  producing  an 
efficient  set  of  indicators,  however  it  can  also  hinder  assessors  by  adding  a  factor  of 
uncertainty  to  the  assessment  effort. 

When  the  desired  effects  have  a  cumulative  relationship  as  is  the  case  above,  the 
assessors  can  leam  the  states  of  multiple  systems  with  fewer  indicators,  saving  time  and 
focusing  valuable  intelligence  resources.  However,  if  the  desired  effects  are  not 
components  of  the  same  causal  linkage  or  the  same  chain  of  effects,  the  same  indicator 
used  as  evidence  for  states  of  multiple  systems  causes  the  indicator  to  be  worth  less  as 
evidence  of  an  individual  system  state.  This  is  due  to  the  multiple  factors  contributing  to 
the  indicator  from  multiple  lines  of  effects.  As  with  the  Jones  Criteria  (reference  Section 
2.6),  the  weights  of  the  indicators  depend  on  their  relative  occurrence  as  a  result  of  each 
of  the  indirect  effects.  For  example,  an  indicator  that  is  anticipated  to  appear  as  result  of 
an  effect  on  system  A,  and  only  system  A,  is  given  more  weight  as  evidence  of  that  effect 
than  an  indicator  that  could  appear  as  a  result  of  an  effect  on  system  A  or  as  a  result  of  an 
effect  on  system  B.  Therefore,  while  having  the  same  indicators  represent  evidence  for 
states  of  multiple  systems  may  appear  to  be  more  efficient,  the  information  gained  may 
actually  be  more  uncertain,  because  assessors  do  not  know  the  true  cause  of  the  indicator. 
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Assessors  should  therefore  strive  to  determine  other  unique  indicators  for  each  effect  on 
separate  causal  linkages.  Ideally,  assessors  will  be  able  to  determine  the  states  of  the 
operational-level  systems  with  approximately  50-60  indicators  (reference  Section  2.5).  If 
the  number  of  indicators  greatly  exceeds  the  50-60  range,  a  filtering  process  may  need  to 
be  employed  to  keep  the  assessment  process  manageable. 

Table  4.1  summarizes  the  resulting  indicators  for  Desired  Effects  1.1  and  1.2. 


Table  4.1  List  of  indicators  for  ODF  Desired  Effects  1.1  and  1.2 


IADS  Neutralized 

OL  loses  air  sovereignty/Coalition 
forces  have  freedom  of  movement 

Decrease  in  SAFIRES 

Increased  OL  sortie  rates 

No  SAFIRES 

Civ  a/c  fly  unusual  patterns 

Decrease  in  radar  emissions 

Civ  a/c  fly  unscheduled  flight  plans 

Ballistic  SAFIRES 

Increased  traffic  away  from  WMD/C2 
facilities 

Decrease  in  comm,  emissions 

Decreased  traffic  towards  WMD/C2 
facilities 

No  radar  emissions 

No  SAFIRES 

No  OL  a/c  airborne 

OL  a/c  engage  Coalition  forces 

4.2. 1.3  Phase  III:  System  State  Definition. 

The  initial  states  of  the  OLTO  and  IADS  systems  s°  =  (SqLT0  A^dS)  and  the 
desired  states  of  the  systems  se  =  (sSltoAiads)  A°w  directly  from  the  effects  definition  of 
Phase  I  and  the  effects-tree  diagrams  of  Phase  II.  For  each  effect  of  Objective  1,  the  state 
transitions  are  represented  with  a  three-state  model.  The  indicators  are  partitioned 
accordingly.  As  stated  in  Chapter  3,  this  phase  may  require  a  trial-and-error  approach  to 
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determine  the  appropriate  number  of  states  and  indicators  for  those  states.  Table  4.2  and 
Table  4.3  show  the  states  and  indicators  decided  upon  by  the  assessors  and  planners  in 
this  scenario. 


Table  4.2  Three-state  model  and  indicators  for  ODF  Desired  Effect  1.2 


‘’IADS 

S° 

s1 

s6 

State  Description 

Little  or  no  change  in  IADS 
capability  or  behavior. 

IADS  undergone  a 
significant  change. 

IADS  threat  to  coalition 
forces  neutralized. 

Indicators 

Guided  SAFIRES 

Decrease  in  SAFIRES 

No  SAFIRES 

Decreased  radar  emissions 

Ballistic  SAFIRES 

Decreased  comm,  emissions 

No  radar  emissions 

Table  4.3  Three-state  model  and  indicators  for  ODF  Desired  Effect  1.1 


m 

^OLTO 

S° 

s1 

s6 

State  Description 
(Given  IADS  neutralized) 

OL  a/c  able  and  willing  to 
contest  coalition  forces. 

OL  a/c  contest  Coalition 
forces  indirectly. 

OL  experiences  complete  loss 
of  air  sovereignty  and 
coalition  forces  have  freedom 
of  access  for  follow-on 
persistence  forces. 

Indicators 

Increased  sortie  rates 

Civilian  a/c  engage  in 
unscheduled  or  unexpected 
flights 

No  SAFIRES 

OL  a/c  engage  Coalition 
forces 

Civilian  a/c  fly  unusual  flight 
patterns 

No  airborne  OL  a/c 

Mobile  SAMs  positioned 
near  high-priority  assets 

Coalition  forces  uncontested 
in  OL  airspace 

4.2. 1.4  Actions  Assessment  Preparation. 

The  planners  have  identified  four  actions  (see  Section  4.2. 1.2)  to  be  taken  in  order 
to  achieve  both  desired  effects;  the  desired  effect  on  the  OL  IADS  is  intended  to  have  a 
cumulative  effect  resulting  in  the  desired  effect  on  the  OLTO  airspace.  The  actions 
chosen  are  all  kinetic  strikes  on  the  IADS.  Therefore,  assessors  choose  the  Measures  of 
Performance  (MOP)  to  be  the  percentage  of  targets  destroyed.  The  planners  set  the  target 
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number  for  the  planned  actions — the  required  percentage  of  targets  to  be  destroyed — at 
80  percent.  That  is,  planners  believe  that  at  least  80  percent  of  the  actions  must  be 
completed  in  order  to  achieve  the  desired  effect  on  the  OL  IADS  and  the  OLTO  airspace. 
These  actions  will  be  tracked  as  EBE  commences  to  detennine  their  level  of 
accomplishment. 

4.2. 1.5  Objective  and  Campaign  Assessment  Preparation. 

Assessors  and  planners  develop  an  initial  progress  function  based  on  the  planned 
COA  and  anticipated  level  of  objective  attainment.  The  plan-line  for  the  COA  and 
anticipated  objective  achievement  is  shown  in  Figure  4.3.  The  initial  progress  function  is 
depicted  in  Figure  4.4.  For  both  examples  presented  in  Chapter  4,  it  is  implied  that  the 
current  system  states,  effects  achievement,  objective  attainment,  and  task 
accomplishment  will  each  be  assessed  following  the  campaign  day  on  which  the  actions 
took  place.  To  aid  discussion,  this  thesis  will  refer  to  the  assessments  made  after  the 
actions  on  day  D+x  as  assessments  at  D+x. 

For  example,  in  Figure  4.3,  the  state  vector  (1,1),  which  corresponds  to 
(5olto’5iads)  ’ 's  anticipated  to  be  reached  at  D+0  (after  the  actions  of  D+0).  Planners 
determine  that  state  vector  (1,1)  will  yield  approximately  80  percent  of  the  overall 
objective  attainment.  State  vector  (2,2),  which  corresponds  to  (^olto’^iads)  > 's 
anticipated  to  be  reached  at  D+l  (after  actions  on  D+l)  and  yield  100-percent  attainment 
of  the  objective. 
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Percent  of  Objective  Attainment 


100% 


Time 


Figure  4.3  Planned  Progress  Line  of  ODF  Objective  1 


100% 


Figure  4.4  Initial  Progress  Function  for  ODF  Objective  1 
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4.2.2  ODF  Objective  2. 


The  first  three  phases  along  with  actio ns-assessment  preparation  and  objective 
and  campaign-assessment  preparation  are  presented  for  Objective  2:  Stop  WMD 
activities  of  OL  leadership. 

4.2.2. 1  Phase  I:  Effects  Definition. 

For  Objective  2  (Stop  WMD  activities  of  OL  leadership),  the  planners  define 
three  desired  effects. 

Desired  Effect  2.1: 

•  System  of  interest:  Orangeland  leadership. 

•  Initial  State  of  the  System:  OL  denies  all  WMD  accusations  and  unwilling  to 
consider  diplomatic  negotiations.  OL  willing  to  use  TBM/WMD  rather  than  lose 
all  of  their  current  WMD  capability  as  a  result  of  coalition  attacks;  “use  or  lose” 
mindset. 

•  Desired  End  State:  OL  leadership  views  WMD  assets  as  “at  risk”  with  a 
moderate  likelihood  of  losing  all  WMD  capability.  OL  would  rather  negotiate 
tenns  for  disanning  WMD  capability  and  ceasing  WMD  ambitions  than  lose  any 
more  capability;  “negotiate  or  lose”  mindset. 

•  Start  time:  Effect  to  begin  on  or  before  D+5. 

•  End  time:  Indefinite;  effect  to  be  maintained  as  long  as  required  by  CFC. 

Desired  Effect  2.2: 

•  System  of  interest:  OL  WMD  Research  and  Development  (R&D)  facilities. 
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•  Initial  State  of  the  System :  Seven  active  chemical  and  biological  R&D  facilities 
and  one  nuclear  R&D  facility  located  throughout  the  country  are  currently 
operating  at  full  capacity. 

•  Desired  End  State:  No  chemical/biological  weapon  R&D  facilities  operating  at 
full-capacity.  Facilities  not  to  sustain  irreparable  damage;  the  operating  level  of 
all  the  facilities  is  to  be  degraded  to  less-than-full  capacity. 

•  Start  time:  Effect  to  begin  on  or  before  D+l . 

•  End  time:  Effect  not  to  end  before  D+10. 

Desired  Effect  2.3: 

•  System  of  interest:  OL  WMD  deployment  systems. 

•  Initial  State  of  the  System:  Three  deployment  facilities  have  the  capability  to 
place  WMD  warheads  on  TBMs. 

•  Desired  End  State:  These  facilities  physically  and  functionally  destroyed. 

•  Start  time:  Effect  to  begin  on  or  before  D+4. 

•  End  time:  Effect  not  to  end  before  D+10. 

4.2.2.2  Phase  II:  Indicator  Development. 

Planners  determine  three  actions  to  be  taken  in  order  to  achieve  a  tactical  effect 
(Electrical  power  disrupted  at  WMD  R&D  facilities)  that  will  achieve  Desired  Effect  2.2 
(Production  disrupted  at  WMD  R&D  facilities): 

■  Disrupt  Petroleum,  Oil  and  Lubrication  (POL)  lines  for  backup  EP  outlets. 

■  Disrupt  backup  EP  generators. 

■  Disrupt  Electrical  Power  (EP)  substations. 
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Figure  4.5  depicts  the  effects-tree  diagram  used  to  develop  a  set  of  measurable  indicators 
for  Desired  Effect  2.2.  This  phase  results  in  seven  measurable  indicators  (as  shown  in 
Table  4.4)  that  will  be  used  to  detennine  the  state  of  the  WMD  R&D  facilities. 

For  Desired  Effect  2.3,  the  planners  determine  four  actions  and  two  tactical 
effects  required  to  achieve  “WMD  deployment  capability  disrupted.” 

■  Destroy  relay  stations. 

■  Disrupt  OL  telecommunications  (tactical  effect). 

■  Destroy  fiber  optic  cable  lines  for  EP  plants. 

■  Disrupt  OL  Command,  Control,  Communications,  and  Computer  (C4)  systems  for 
EP  (tactical  effect). 

■  Destroy  WMD  storage  facilities. 

■  Destroy  POL  for  WMD  units. 

This  effects-tree  diagram  pictured  in  Figure  4.6  results  in  seven  measurable  indicators 
that  will  be  used  to  determine  the  state  of  the  WMD  deployment  facilities.  The  effects- 
tree  diagram  also  reveals  four  potential  indicators  that  assessors  deem  unmeasurable, 
indicated  by  the  dashed  lines.  Recall  from  Chapter  3  that  an  indicator  must  be  observable 
and  distinguishable  in  order  to  be  measurable.  Note  that  the  indicators  developed  for 
Desired  Effects  2.2  and  2.3  connote  that  ISR  assets  should  be  focused  on  the  lines  of 
communication  (LOC)  associated  with  the  WMD  R&D  and  deployment  facilities. 

Desired  Effect  2.1  is  a  cumulative  effect  resulting  from  the  achievement  of 
Desired  Effects  2.2  and  2.3.  The  effects-tree  diagram  pictured  in  Figure  4.7  yields  12 
measurable  indicators  for  Desired  Effect  2.1.  These  indicators  involve  a  variety  of 
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systems,  including  LOCs,  military  defense  systems,  diplomatic  channels,  TBM  launch 
systems,  and  international  terrorist  activities. 


Disrupt  POL  for 

backup  EP  outlets.  Disrupt  backup  EP 
generators 


Disrupt  EP 
substations. 


Tactical  effect  leads  to  achievement  of 
desired  effect  2.2. 


WMD  R&D  facilities 
at  operating  at  full 
capacity. 


OL  moves  WMD 
materials 


Shut  down  facility 


► 


Increased  traffic 
to  and  from 
facilities 


Decreased  traffic 
to  facilities 


*• 


Production  at 
WMD  R&D 
facilities 
disrupted. 


Local  blackout 
around  R&D 
facilities 


Lights  stay  out  at  R&D 
facilities  while  power 
comes  back  to 
surrounding  area 

Mobile  SAM 
located  IVO 


facilities 

Evacuate  facility 

Decreased 
traffic  to 
facilities 

Increased  security^ 

Safety  systems 

Evacuate  local 
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Figure  4.5  Effects-tree  diagram  for  ODF  Desired  Effect  2.2 
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Figure  4.6  Effects-tree  diagram  for  ODF  Desired  Effect  2.3 
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Table  4.4  summarizes  the  indicators  developed  in  Phase  II  for  Desired  Effects  2.1,  2.2, 


and  2.3. 


Table  4.4  List  of  indicators  for  ODF  Desired  Effects  2.1,  2.2,  and  2.3 


DE  2.1  OL  leadership  views 
WMD  “at  risk;”  would  rather 
negotiate  than  lose  all  WMD 
assets 

DE  2.2  Production  at  WMD 
R&D  facilities  disrupted 

DE  2.3  WMD  deployment 
capability  disrupted 

Increased  heavy  equipment  traffic 
in/out  of  WMD  facilities 

Decreased  traffic  to  facilities 

Increased  heavy  equipment  traffic 
at  WMD  deployment  sites 

Military  defenses  relocated  to 

WMD  R&D  facilities 

Increased  heavy-equipment  traffic 
to  and  from  facilities 

Fires  and  explosions  at  WMD 
deployment  sites 

OL  emplaces  civilians  I VO  WMD 
sites 

Local  blackout  around  R&D 
facilities 

Increased  traffic  out  of/  decreased 
traffic  to  WMD  deployment  sites 

OL  publicizes  civilian 
consequences  (blackouts  to 
hospitals/schools,  casualties  etc.) 

Lights  stay  out  at  R&D  facilities 
while  power  returns  to  surrounding 
area 

OL  forces  evacuate  WMD 
deployment  sites* 

OL  issues  a  demarche  against  US 

Explosions,  fires  at  R&D  facilities 

OL  forces  garrison  at  WMD 
deployment  sites* 

OL  initiates  diplomacy  via  third 
party 

Increased  security  at  WMD  R&D 
facilities 

Increased  cell  phone  use 

OL  launches  TBM  into  own 
territory 

Mobile  SAM  IVO  WMD  R&D 
facilities 

Increased  radio  use 

OL  launches  TBM  at  Coalition 
forces 

Increased  SATCOM  use 

OL  launches  TBM  at  neighboring 
countries 

OL  use  courier  to  communicate* 

Increased  terrorist  attacks  in 
US/Coalition  allies 

Unexplained  civilian  casualties* 

Terrorist  attacks  in  neighboring 
countries 

OL  civilian  a/c  used  as  WMD 
(flying  irregular  pattems/times) 

*  unmeasurable  indicator 

4.2.2.3  Phase  III:  System  State  Definition. 

The  initial  states  of  the  systems  and  the  desired  states  of  the  systems  flow  directly 
from  the  effects  definition  and  the  effects-tree  diagrams.  The  state  transitions  for  Desired 
Effect  2.1,  2.2,  and  2.3  are  represented  by  a  four-state  model,  a  two-state  model,  and  a 
three-state  model  respectively.  Tables  4.5,  4.6,  and  4.7  show  the  states  and  indicators 
decided  upon  by  the  assessors  and  planners. 
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Table  4.5  State  Transitions  and  Indicators  for  ODF  Desired  Effect  2.1 


m 

^OL 

s° 

s1 

s2 

State  Description 

Little  or  no  change  to 
system  function. 

System  functionality 
affected. 

Little  bit  better.. 

System  unable  to 
perform  mission. 

Indicators 

OL  launches  TBM  at 
Coalition  forces 

OL  civilian  a/c  used 
as  WMD  (flying 
irregular 
pattems/times) 

OL  emplaces 
civilians  I VO 
WMD  sites 

OL  issues  a  demarche 
against  US 

OL  launches  TBM  at 
neighboring  countries 

OL  publicizes 
civilian  consequences 
(blackouts  to 
hospitals/schools, 
casualties  etc.) 

Military  defenses 
relocated  to 

WMD  R&D 
facilities 

OL  initiates  diplomacy 
via  third  party 

Increased  terrorist 
attacks  in  US/Coalition 
allies 

OL  launches  TBM 
into  own  territory 

Increased  heavy 
equipment  traffic 
in/out  of  WMD 
facilities 

Terrorist  attacks  in 
neighboring  countries 

Table  4.6  State  Transitions  and  Indicators  for  ODF  Desired  Effect  2.2 


m 

^  WMD  R&D 

s° 

State  Description 

Little  or  no  change  to  system 
function. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy-equipment 
traffic  to  and  from  facilities 

Local  blackout  around  R&D 
facilities 

Increased  security  at  WMD 
R&D  facilities 

Lights  stay  out  at  R&D 
facilities  while  power  returns 
to  surrounding  area 

Mobile  SAM  IVO  WMD 

R&D  facilities 

Decreased  traffic  to  facilities 

Explosions,  fires  at  R&D 
facilities 

4.2.2.4  Actions  Assessment  Preparation. 

The  planners  have  identified  a  total  of  seven  actions  (see  Section  4.2. 2. 2)  to  be 
taken  in  order  to  achieve  the  three  desired  effects  associated  with  Objective  2.  Again,  the 
actions  are  all  kinetic  strikes.  Therefore,  assessors  choose  the  percentage  of  the  targets 
destroyed  as  the  MOPs  for  the  actions.  Due  to  the  precision  nature  of  the  planned 
friendly  actions,  the  planners  set  the  target  number  for  the  planned  actions  at  100  percent. 
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Every  specified  target  needs  to  be  successfully  acted  upon  to  achieve  the  desired  effect. 
Note  that  the  WMD  R&D  facilities  are  not  targeted  directly;  only  the  support  systems 
feeding  the  R&D  facilities,  illustrating  the  precision  nature  of  the  strikes.  These  actions 
will  be  tracked  as  EBE  commences  to  detennine  their  level  of  accomplishment. 


Table  4.7  State  Transitions  and  Indicators  for  ODF  Desired  Effect  2.3 


m 

^  WMD  Dep 

s° 

sl 

State  Description 

Little  or  no  change  to  system 
function. 

System  functionality  affected. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy  equipment 
traffic  at  WMD  deployment 
sites 

Increased  cell  phone  use 

Fires  and  explosions  at  WMD 
deployment  sites 

Increased  traffic  out  of/ 
decreased  traffic  to  WMD 
deployment  sites 

Increased  radio  use 

OL  forces  evacuate  WMD 
deployment  sites* 

OL  forces  garrison  at  WMD 
deployment  sites* 

Increased  SATCOM  use 

OL  use  courier  to 
communicate* 

unmeasurable  indicator 


4.2.2.5  Objective  Assessment  Preparation. 

Assessors  and  planners  next  develop  an  initial  progress  function  based  on  the 
planned  COA  and  anticipated  level  of  objective  attainment.  The  plan-line  for  the  COA 
and  anticipated  objective  achievement  for  Objective  2  is  shown  in  Figure  4.8.  The  initial 
progress  function,  derived  from  the  planned  progress  line,  is  depicted  in  Figure  4.9. 

The  state  vector  (s*,  ^MDRSD,ilD[p)  represents  the  states  of  the  systems  of 

interest  and  is  translated  by  the  following  vectors  in  the  planned  progress  line  and 
progress  function:  the  state  vector  (0,1,1)  is  anticipated  to  be  reached  at  D+0  and  yield 
approximately  20  percent  of  the  overall  objective  attainment.  State  vector  (1,1,1)  is 
anticipated  to  be  reached  at  D+2  yielding  30  percent  of  objective  attainment.  State  vector 
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(0,1,2)  is  anticipated  to  be  reached  at  D+3.  This  state  vector  represents  full  achievement 
of  Desired  Effect  2.2,  partial  achievement  of  Desired  Effect  2.3,  and  no  achievement  of 
Desired  Effect  2.1,  which  is  a  return  to  the  initial  state  of  the  OL  leadership.  This  plan 
results  from  the  planners’  forecast  that  OL  will  have  an  undesired  reaction  to  coalition  air 
strikes  and  will  likely  fire  a  TBM  at  its  neighbors.  Per  the  effect  definitions,  state  vector 
(3,1,2),  representing  the  achievement  of  all  the  desired  effects  associated  with  this 
objective,  is  anticipated  to  be  reached  at  D+4  and  yield  100-percent  attainment  of  the 
objective. 


Figure  4.8  Planned  Progress  of  ODF  Objective  2 
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100% 


State  Vectors 

Figure  4.9  Initial  Progress  Function  for  ODF  Objective  2 

4.2.3  Post  Effects-Based  Execution  of  ODF. 

To  describe  the  assessment  methodology  more  fully,  this  section  details  the  EBA 
process  as  EBE  commences.  Chapter  4  examines  the  ODF  scenario  at  D+0  (D-Day)  and 
at  D+4  with  the  aim  of  showing  enough  assessment  detail  to  demonstrate  the 
methodology.  For  the  interested  reader,  the  observed  indicators  and  daily  cumulative 
actions  accomplishment  for  the  remaining  campaign  days  are  given  in  Appendix  B. 

4.2.3. 1  Phase  IV:  Actions  Assessment. 

At  D+0,  strike  results  are  mixed.  Physical,  functional,  and  system  assessments 
reveal  that  the  overall  actions  accomplishment  is  about  60  percent.  Figure  4.10  lists  the 
accomplishment  of  each  action  individually.  As  stated  above,  the  planners  determined 
that  approximately  80  percent  of  the  IADS  targets  needed  to  be  successfully  serviced  to 
achieve  Desired  Effects  1.1  and  1.2.  Similarly,  to  achieve  Desired  Effects  2.1,  2.2,  and 
2.3,  all  targets  associated  with  the  WMD  R&D  electrical  power  and  WMD  deployment 
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facilities  need  to  be  successfully  serviced.  Therefore,  at  D+0,  coalition  forces  are  closer 
to  accomplishing  the  actions  associated  with  the  IADS  systems  than  they  are  to 
accomplishing  the  actions  associated  with  the  WMD  and  OL  systems. 


4.23.2  Phase  V:  Effects  Assessment. 


At  D+0,  some  planned  indicators  are  not  being  seen  at  all  and  some  other  planned 
indicators  are  returning  ambiguous  information.  Tables  4.8  through  4.12  highlight  in 
gray  the  indicators  triggered  by  the  available  intelligence  data.  Table  4.8  (Desired  Effect 
1 .2)  illustrates  that  no  SAFIRES  were  reported  during  the  first  day  of  air  strikes, 
indicating  that  the  IADS  system  might  already  be  in  the  desired  state. 
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Figure  4.10  ODF  Actions  Accomplishment  at  D+0 
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Table  4.8  Desired  Effect  1.2,  IADS  Indicators  at  D+0 


m 

^  IADS 

A IADS 

1 

^  IADS 

ce 

^  IADS 

State  Description 

Little  or  no  change  in 
IADS  capability  or 
behavior. 

IADS  undergone  a 
significant  change. 

IADS  threat  to 
coalition  forces 
neutralized. 

Guided  SAFIRES 

Decrease  in  SAFIRES 

No  SAFIRES 

Indicators 

Decreased  radar 
emissions 

Ballistic  SAFIRES 

Decreased  comm, 
emissions 

No  radar  emissions 

Table  4.9  (Desired  Effect  1.1)  illustrates  similar  indications  (no  SAFIRES,  coalition 
forces  uncontested  in  OL  airspace,  and  no  OL  aircraft  attempt  to  get  airborne),  placing 
OLTO  airspace  in  the  desired  state  as  well. 


Table  4.9  Desired  Effect  1.1,  OLTO  Airspace  Indicators  at  D+0 


m 

^  OLTO 

A OLTO 

e1 

A OLTO 

ee 

^  OLTO 

State  Description 
(Given  IADS 
neutralized) 

OL  a/c  able  and  willing 
to  contest  coalition 
forces. 

OL  a/c  contest 

Coalition  forces 
indirectly. 

OL  experiences 
complete  loss  of  air 
sovereignty  and 
coalition  forces  have 
freedom  of  access  for 
follow-on  persistence 
forces. 

Indicators 

Increased  sortie  rates 

Civilian  a/c  engage  in 
unscheduled  or 
unexpected  flights 

No  SAFIRES 

OL  a/c  engage 

Coalition  forces 

Civilian  a/c  fly  unusual 
flight  patterns 

No  OL  a/c  fly  w/o 
explicit  approval  from 
Coalition  forces 

Increased  traffic  away 
from  WMD/C2 
facilities 

Coalition  forces 
uncontested  in  OL 
airspace 

Decreased  traffic 
towards  WMD/C2 
facilities 

No  airborne  OL  a/c 
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In  Table  4. 10  (Desired  Effect  2.2),  assessors  see  the  desired  state  se  for  the  WMD  R&D 
facilities  with  two  indicators  of  that  state,  a  local  blackout  around  the  facilities  and  power 
returning  to  surrounding  areas  but  not  the  R&D  facilities. 


Table  4.10  Desired  Effect  2.2,  WMD  R&D  Facility  Indicators  at  D+0 


m 

°  WMD  R&D 

S° 

S6 

State 

Description 

Little  or  no  change  to  system 
function. 

System  unable  to  perform  mission. 

Indicators 

increased  heavy-equipment  traffic  to 
and  from  facilities 

Local  blackout  around  R&D  facilities 

Increased  security  at  WMD  R&D 
facilities 

Lights  stay  out  at  R&D  facilities  while 
power  returns  to  surrounding  area 

Mobile  SAM  IVO  WMD  R&D 
facilities 

Decreased  traffic  to  facilities 

Explosions,  fires  at  R&D  facilities 

In  Table  4.1 1  (Desired  Effect  2.3),  assessors  have  witnessed  indicators  of  two  states,  s1 
and  ,s,c.  Neither  state  has  the  prerequisite  40  percent  of  indicators  observed,  therefore  the 
assessors  must  carefully  consider  the  implications  of  the  indicators.  Using  the  guidelines 
for  detennining  the  system  states,  the  assessors  conclude  that  the  system  is  most  likely  in 
the  desired  state,  however  since  less  than  60  percent  of  the  actions  against  this  system 
have  been  accomplished,  the  planners  decide  to  declare  the  system  in  s1. 

Table  4.12  (Desired  Effect  2.1)  illustrates  that  OL  launched  TBMs  at  both 
Coalition  forces  and  OL  neighboring  countries,  placing  the  OL  leadership  in  the  initial 
state  5°. 
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Table  4.11  Desired  Effect  2.3,  WMD  Deployment  Capability  Indicators  at  D+0 


S z' 

WMD  dep 

s1 

State  Description 

Little  or  no  change  to 
system  function. 

System  functionality 
affected. 

System  unable  to 
perform  mission. 

Indicators 

Increased  heavy 
equipment  traffic  at 
WMD  deployment  sites 

Increased  cell  phone 
use 

Fires  and  explosions  at 
WMD  deployment  sites 

Increased  traffic  out  of/ 
decreased  traffic  to 

WMD  deployment  sites 

Increased  radio  use 

OL  forces  evacuate 
WMD  deployment 
sites* 

OL  forces  garrison  at 
WMD  deployment 
sites* 

Increased  SATCOM 

use 

No  activity  at  WMD 
deployment  sites. 

OL  use  courier  to 
communicate* 

Table  4.12  Desired  Effect  2.1,  OL  Leadership  Indicators  at  D+0 


m 

dOL 

1 

s 

2 

s 

State  Description 

Little  or  no  change 
to  system  function. 

System 

functionality 

affected. 

Little  bit 

worse. 

System  unable  to 
perform  mission. 

Indicators 

Increased  heavy 
equipment  traffic 
in/out  of  WMD 
facilities 

Military 
defenses 
relocated  to 
WMD  R&D 
facilities 

OL  emplaces 
civilians  IVO 
WMD  sites 

OL  issues  a 
demarche  against 

US 

OL  publicizes 
civilian 
consequences 
(blackouts  to 
hospitals/schools, 
casualties  etc.) 

OL  civilian 
a/c  used  as 
WMD  (flying 
irregular 
patterns/times) 

OL  launches 
TBM  into  own 
territory 

OL  initiates 
diplomacy  via 
third  party 

OL  launches  TBM 
at  Coalition  forces 
OL  launches  TBM 
at  neighboring 
countries 

Increased  terrorist 
attacks  in 

US/Coalition  allies 
Terrorist  attacks  in 
neighboring 
countries 
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4.2.3.3  Phases  VI  and  VII:  Objective  and  Campaign  Assessment. 

After  considering  all  the  system  states,  the  assessors  detennine  the  state  vectors  at 
D+0  to  be  (2,2)  for  Objective  1  and  (0,1,1)  for  Objective  2.  Both  state  vectors  have 
already  been  mapped  to  progress  values  via  the  progress  function  (reference  Sections 
4.2. 1 .5  and  4. 2.2. 5).  Figure  4.11  and  Figure  4. 12  illustrate  the  actual  progress  line  versus 
the  planned  progress  line  at  D+0  for  Objectives  1  and  2. 


Figure  4.11  Actual  Progress  Line  for  Objective  1  at  ODF  D+0 


This  scenario  now  jumps  ahead  to  the  assessments  at  D+4.  The  cumulative 
actions  accomplishment,  current  progress  functions,  and  progress  curves  are  shown  as 
they  exist  at  D+4  in  order  to  illustrate  how  the  assessment  of  ODF  has  changed  over  the 
first  few  days  of  the  air  campaign. 
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Figure  4.12  Actual  Progress  Line  for  Objective  2  at  ODF  D+0 


Figure  4.13  depicts  the  levels  of  action  accomplishment  at  D+4.  Of  the  eleven 
actions,  only  three  have  reached  their  assigned  target  levels.  The  effects  of  these  actions 
are  shown  in  Figure  4.14  and  Figure  4.15.  The  progress  function  for  Objective  1  (Figure 

4.14)  has  been  updated  as  the  state  vectors  (0,1)  and  (1,2)  have  been  observed  via  the 
indicators.  Objective  1  returns  to  full  attainment  at  D+4  having  previously  dipped  to  only 
30  percent  attainment  at  D+2.  The  updated  progress  function  for  Objective  2  (Figure 

4.15)  contains  the  state  vectors  that  have  been  observed.  All  of  the  state  vectors  mapped 
to  progress  values  by  the  planners  are  listed  in  increasing  order  of  progress  as  follows: 
(0,0,0),  (0,1,1),  (1,0,2),  (1,1,0),  (1.1.1),  (0,1,2),  (2,1,1),  and  (3,1,2).  Objective  2  has  not 
progressed  as  positively  as  Objective  1  with  an  approximate  progress  of  just  over  20 
percent  objective  attainment.  Though  the  actions  have  not  been  fully  accomplished  for 
this  objective,  they  are  close  and  it  appears  that  they  are  not  having  the  desired  effects. 
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The  planners  must  take  this  into  consideration  when  deciding  on  the  next  actions  to 
execute. 

At  D+4  it  appears  that  Objective  1  is  on  plan  while  Objective  2  is  lagging.  The 
CFACC  and  planners  will  take  this  into  consideration  for  future  COAs.  As  the  air 
campaign  progresses,  the  progress  function  and  progress  line  will  continue  to  be  updated 
according  to  the  assessment  of  the  effects. 
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Figure  4.1 3  Task  Accomplishment  at  D+4 
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Figure  4.14  ODF  Objective  1  Progress  Function  and  Progress  at  D+5 


Figure  4.15  ODF  Objective  2  Progress  Function  and  Progress  at  D+5 


4.3  Operation  CLOSE  GAP 

Before  the  dictatorial  rule  of  Syrpentor  in  Exstan,  the  Exstani  education  system 
was  a  highly  regarded  learning  institution.  Under  the  Syrpentor  regime,  public  funds 
were  siphoned  off  for  military  expenditures  and  other  regime  priorities.  This  shortage  of 
funds,  combined  with  the  politicization  of  the  education  system,  which  enforced  strict 
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control  over  the  curriculum,  teachers,  and  administration,  propelled  the  education  system 
into  a  steady  decline.  The  recent  conflict  that  liberated  the  Exstani  people  compounded 
twenty  years  of  neglect,  leaving  the  education  system  virtually  defunct.  Nearly,  all 
primary  and  secondary  educational  facilities  require  significant  reconstruction.  As 
Exstani  governance  moves  towards  a  stable  democracy,  the  demands  on  the  US- lead 
coalition  remain  great.  As  a  part  of  an  overall  campaign  of  stability  operations,  the 
Coalition  has  the  following  operational  objective  for  the  education  system: 

Objective  1 :  Improve  quality  and  access  to  education. 

4.3.1  Phase  I:  Desired  Effect  Definition. 

For  Objective  1,  the  planners  define  two  desired  effects. 

Desired  Effect  1 : 

•  System  of  interest :  Exstani  school-age  children. 

•  Initial  State  of  the  System :  Few  students  attend  school  regularly  due  to  limited 
availability  of  schools/teachers  and  limited  access  to  school  supplies. 

•  Desired  End  State :  National  education  enrollment  and  attendance  returned  to  pre¬ 
regime  levels. 

•  Start  time :  24  months  from  start  of  program. 

•  End  time :  Indefinite. 

Desired  Effect  2: 

•  System  of  interest :  Exstan  Education  Infrastructure. 

•  Initial  State  of  the  System :  Educational  facilities  at  all  levels  are  in  shambles  and 
are  in  dire  need  of  repair.  Few  schools  are  open  full-time  due  to  lack  of  qualified 
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teachers  and  administrators.  The  schools  that  are  open  have  no  common  text 
books  or  other  learning  materials. 

•  Desired  End  State :  Infrastructure  meets  basic  standards  and  provides  basic 
educational  supplies  and  tools  for  all  schools  nationwide. 

•  Start  time :  24  months  from  start  of  program. 

•  End  time :  Indefinite. 

4.3.2  Phase  II:  Indicator  Development. 

In  Phase  II,  assessors  analyze  Desired  Effect  2  before  Desired  Effect  1  because 
Desired  Effect  2  is  expected  to  have  a  cumulative  effect  contributing  to  Desired  Effect  1 . 
Planners  detennine  seven  actions  to  be  taken  in  order  to  achieve  Desired  Effect  2: 

■  Rehabilitate  substandard  schools. 

■  Build  new  schools. 

■  Provide  all  schools  with  potable  water  and  sanitation  facilities. 

■  Train  new  teachers  and  administrators. 

■  Distribute  desks  and  chalkboards  to  all  schools. 

■  Distribute  new  math  and  science  text  books  to  all  schools. 

■  Distribute  sports  equipment  to  all  schools. 

Figure  4.16  depicts  the  effects-tree  diagram  used  to  develop  a  set  of  measurable 
indicators  for  Desired  Effect  2.  Multiple  actions  are  anticipated  to  result  in  similar 
system  reactions.  The  effects-tree  analysis  results  in  19  unique  indicators  that  will  be 
used  to  detennine  the  state  of  the  education  infrastructure  (shown  in  Table  4.13). 
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Using  a  similar  process,  Figure  4.17  illustrates  the  effects-tree  diagram  used  to 
determine  measurable  indicators  for  Desired  Effect  1.  To  build  this  effects-tree  diagram, 
planners  examine  the  COA,  which  includes  four  actions  to  be  taken  and  one  operational 
effect  to  be  achieved  in  order  to  achieve  Desired  Effect  1 : 

■  Establish  model  schools  as  “Centers  of  Excellence.” 

■  Enroll  out-of-school  children  in  accelerated  programs. 

■  Develop  and  broadcast  early  childhood  television  series  promoting  education. 

■  Education  Infrastructure  meets  basic  standards  and  provides  basic  educational 
supplies  and  tools  for  all  schools  nationwide  (Desired  Effect  2). 

Again,  multiple  actions  are  anticipated  to  result  in  similar  system  reactions.  This  effects- 
tree  analysis  results  in  13  unique  indicators  that  will  be  used  to  determine  the  state  of  the 
Exstani  school-age  children  (shown  in  Table  4.13). 

This  example  illustrates  two  advantages  over  assessing  combat  operations.  First, 
there  is  no  enemy  per  se  trying  to  deceive  or  conceal  capabilities  from  coalition  forces. 
Therefore,  the  systems  needing  to  be  monitored  in  order  to  measure  the  indicators  are 
assumed  to  be  easier  to  track.  Second  (and  somewhat  related),  the  desired  effects  in  this 
example  lend  themselves  to  being  measured  directly.  That  is,  the  best  indicators  of  the 
desired  effects  are  the  systems  of  interest  themselves.  For  example,  in  Desired  Effect  1 
(enrollment  and  attendance  return  to  pre -regime  levels),  assessors  can  track  the  number 
of  students  enrolled  directly  in  order  to  determine  the  state  of  the  system.  Likewise,  for 
Desired  Effect  2  (infrastructure  meets  basic  standards),  assessors  can  track  the 
educational  facilities  directly  to  determine  the  state  of  the  education  infrastructure. 
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Figure  4.16  Effects-Tree  Diagram  for  OCG  Desired  Effect  2 


130 


Figure  4.17  Effects-Tree  Diagram  for  OCG  Desired  Effect  1 
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4.3.3  Phase  III:  System  State  Definition. 


The  initial  states  of  the  systems  and  the  desired  states  of  the  systems  flow  directly 
from  the  effects  definition  and  the  effects-tree  diagrams.  Assessors  aim  to  develop  a 
state-transition  model  where  each  state  has  three  to  four  indicators.  This  will  provide  the 
property  of  the  model  that  at  least  two  indicators  need  to  be  witnessed  to  meet  the  40- 
percent  rule.  In  this  example,  assessors  represent  the  state  transitions  for  Desired  Effect  1 
and  2  with  a  three-state  model  and  a  four-state  model  respectively.  In  this  example,  the 
number  of  indicators  developed  would  allow  the  assessors  to  model  the  systems  with 
more  states  if  necessary.  Tables  4.14  and  4.15  show  the  states  and  indicators  decided 
upon  by  the  assessors  and  planners.  Again,  assessors  see  that  some  indicators  are 
evidence  for  multiple  systems.  Since  the  effects  in  OCG  are  along  the  same  line  of 
effects  (one  desired  effect  is  to  result  as  a  cumulative  effect  of  another)  this  provides  the 
assessors  with  some  efficiency  in  the  set  of  indicators. 

4.3.4  Actions  Assessment  Preparation. 

The  planners  have  identified  1 1  actions  to  be  taken  and  one  operational  desired 
effect  (see  Section  4.3.2)  to  be  achieved  in  order  to  achieve  both  desired  effects.  The 
desired  effect  on  the  Educational  Infrastructure  is  intended  to  have  a  cumulative  effect 
resulting  in  the  desired  effect  on  the  Exstani  school-aged  children.  After  examining  the 
actions,  the  assessors  determine  the  MOPs  to  be  the  following: 

■  Percentage  of  required  substandard  schools  rehabilitated. 

■  Percentage  of  required  new  schools  built. 

■  Percentage  of  schools  with  potable  water  and  sanitation  facilities. 
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■  Percentage  of  required  new  teachers  and  administrators  trained. 

■  Percentage  of  required  desks  and  chalkboards  distributed. 

■  Percentage  of  required  new  math  and  science  text  books  distributed. 

■  Percentage  of  required  sports  equipment  distributed. 

■  Percentage  of  required  “Centers  of  Excellence”  completed. 

■  Percentage  of  required  out-of-school  children  enrolled  in  accelerated  programs. 

■  Early  childhood  television  series  broadcasting  on  Exstani  television  stations. 

■  Education  Infrastructure  (Desired  Effect  2)  measured  via  the  effects  assessment. 
The  planners  set  the  target  number  for  the  planned  actions  at  100  percent.  That  is, 
planners  believe  that  all  planned  actions  must  be  accomplished  completely,  meeting  all 
required  numbers  in  order  to  achieve  the  desired  effects.  These  actions  will  be  tracked  as 
EBE  commences  to  detennine  their  level  of  accomplishment. 

4.3.5  Objective  and  Campaign  Assessment  Preparation. 

Assessors  and  planners  next  develop  a  progress  function  based  on  the  planned 
actions  and  anticipated  levels  of  objective  attaimnent.  The  planned  progress  line  for  the 
planned  COA  is  shown  in  Figure  4.18.  The  initial  progress  function  is  depicted  in  Figure 
4.19. 

The  state  vector  (1,1),  which  corresponds  to  (Students  >  JL*ructme)  > is  anticipated  to 
be  reached  six  months  after  the  program  begins  and  yield  approximately  20  percent  of  the 
overall  objective  attainment.  State  vector  (2,2),  which  corresponds  to  (^students ^infrastructure)  > 
is  anticipated  to  be  reached  18  months  after  the  program  begins  and  yield  approximately 


133 


60  percent  attainment  of  the  objective.  The  full  attainment  of  the  objective,  state  vector 
(2,3),  is  expected  to  be  reached  24  months  after  the  program  begins. 


Table  4.13  Indicators  for  OCG  Desired  Effectsl  and  2 


DE  1  Enrollment  and  attendance  to 
pre-war  levels. 

DE  2  Infrastructure  is  sound. 

Administrators/teachers  hold  workshops 
and  seminars  to  share  best  practices 

Schools  hiring  maintenance  staff 

Increase  in  number  of  teacher  and 
administrator  applications 

Waiting  lists  develop  for  model  schools 

Increase  in  enrollment 

Schools  open  full  time 

Decrease  in  behavioral  problems 

Schools  used  as  community  centers 

Higher  math  and  science  test  scores 

Increase  in  regular  attendance 

Increase  in  attendance  in  poor 
metropolitan  areas 

Math/science  activities  develop  (fairs, 
competitions,  etc.) 

Decrease  in  dropout  rates 

Increase  in  enrollment  in  rural  areas 

Education  facilities  not  in  use 

Higher  standardized  test  scores 

Decrease  in  crime/vandalism  in  and 
around  schools 

All  Ed  facilities  fully  operational 

Higher  grades 

Interscholastic  athletic  competitions 
develop 

Schools  at  full  capacity  for  students 

Increased  enrollment 

Schools  initiate  sports  programs 

Development  of  sports,  clubs,  and  after¬ 
school  activities. 

Schools  develop  PE  curriculum 

Attendance  at  desired  level 

Decrease  in  preventable  illnesses  in 
children 

Enrollment  at  desired  level 

Increase  attendance  across  the  board 

No  change  or  drop  in  enrollment 

Schools  fully  staffed 

No  change  or  drop  in  regular  attendance 

4.3.6  Post  Effects-Based  Execution. 

The  OCG  scenario  is  examined  in  this  chapter  after  18  months  of  stability 
operations  have  been  executed.  The  actions  and  effects  assessments  after  six  months  and 
12  months  are  available  in  Appendix  C. 
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Table  4.14  State  Transitions  and  Indicators  for  OCG  Desired  Effect  1 


m 

e 

Students 

S° 

sl 

se 

State  Description 

Little  or  no  change 
to  enrollment  and 
attendance. 

Noticeable  increase 
in  enrollment  and 
attendance. 

System  at  desired 
levels. 

Indicators 

Decrease  in  behavioral 
problems 

Development  of  sports, 
clubs,  and  after-school 
activities. 

Enrollment  at  desired 
level. 

Higher  standardized  test 
scores 

Waiting  lists  develop  for 
model  schools 

Attendance  at  desired 
level. 

Higher  grades 

Admini  strators/ teachers 
hold  workshops  and 
seminars  to  share  best 
practices 

Decrease  in  dropout  rates 

No  change  or  drop  in 
enrollment 

Increase  in  regular 
attendance 

No  change  or  drop  in 
regular  attendance 

Increased  enrollment 

Table  4.15  State  Transitions  and  Indicators  for  OCG  Desired  Effect  2 


m 

$ 

Infrastructure 

s° 

51 

s2 

State  Description 

Little  or  no  change  to 
system  capability. 

System  capability 
noticeably  increased. 

System  capability 

significantly 

increased. 

System  able  to 
perform  mission. 

Indicators 

Schools  hiring 
maintenance  staff 

Interscholastic 
athletic  competitions 
develop 

Increase  in 
attendance  in 
poor 

metropolitan 

areas 

Schools  used  as 
community  centers 

Increase  in  number  of 
teacher  and 
administrator 
applications 

Schools  initiate  sports 
programs 

Increase  in 
enrollment  in 
rural  areas 

All  Ed  facilities  fully 
operational 

Decrease  in 
crime/vandalism  in 
and  around  schools 

Schools  develop  PE 
curriculum 

Higher  math  and 
science  test 

scores 

Schools  open  full  time 

Education  facilities  not 
in  use 

Math/science 
activities  develop 
(fairs,  competitions, 
etc.) 

Decrease  in 
preventable 
illnesses  in 
children 

Schools  folly  staffed 

Increase 

attendance  across 
the  board 

Schools  at  full 
capacity  for  students 

Increase  in 
enrollment 
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Figure  4.18  Planned  Progress  of  OCG  Objective  1 
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Figure  4.19  Initial  Progress  Function  for  OCG  Objective  1 
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4.3.7  Phase  IV:  Actions  Assessment. 


Figure  4.20  depicts  the  cumulative  levels  of  action  accomplishment  after  18 
months.  The  actions  are  being  accomplished  as  planned.  Of  the  eleven  actions,  four 
have  effectively  reached  100-percent  accomplishment,  while  the  others  have  all  been 
accomplished  above  80  percent  with  a  mean  accomplishment  of  approximately  91 
percent. 


_  _  _  _  Target  Line 


18  mos 


Figure  4.20  OCG  Actions  Accomplishment  at  18  Months 
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4.3.8  Phase  V:  Effects  Assessment. 


After  18  months  of  stability  operations,  assessors  have  indicators  of  increased 
performance  of  students  in  the  classroom,  the  development  of  extracurricular  activities, 
and  an  increase  of  regular  attendance  of  students,  but  not  increased  enrollment.  The 
infrastructure  appears  to  be  strengthening  as  the  schools  are  fully  staffed,  with  an  increase 
in  qualified  applicants  for  teaching  and  administration  positions,  and  the  need  to  hire 
additional  maintenance  staff.  The  indicators  observed  by  the  assessors  are  given  in 
Tables  4.16  and  4.17. 


Table  4.16  Student  Enrollment  Indicators  at  18  Months 


^  Students 

i 

s 

se 

State  Description 

Little  or  no  change 
to  enrollment  and 
attendance. 

Noticeable  increase 
in  enrollment  and 
attendance. 

System  at  desired 
levels.. 

Indicators 

Decrease  in  behavioral 
problems 

Development  of  sports, 
clubs,  and  after-school 
activities. 

Enrollment  at  desired 
level. 

Higher  standardized  test 
scores 

Waiting  lists  develop  for 
model  schools 

Attendance  at  desired 
level. 

Higher  grades 

Admini  strators/ teachers 
hold  workshops  and 
seminars  to  share  best 
practices 

Decrease  in  dropout  rates 

Increase  in  regular 
attendance 

Increased  enrollment 

The  assessors  see  that  for  Desired  Effect  1,  there  are  two  indicators  of  system  state  s'  and 
one  indicator  for  system  state  5°.  .v1  has  two  out  of  five  indicators  observed;  using  the 
40-percent-rule,  the  assessors  deem  the  student  enrollment  to  be  in  s  . 
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Table  4.17  Education  Infrastructure  at  18  Months 


m 

$ 

Infrastructure 

5° 

51 

s2 

State  Description 

Little  or  no  change  to 
system  capability. 

System  capability 
noticeably  increased. 

System  capability 

significantly 

increased. 

System  able  to 
perform  mission. 

Indicators 

Schools  hiring 
maintenance  staff 

Interscholastic 
athletic  competitions 
develop 

Increase  in 
attendance  in 
poor 

metropolitan 

areas 

Schools  used  as 
community  centers 

Increase  in  number  of 
teacher  and 
administrator 
applications 

Schools  initiate  sports 
programs 

Increase  in 
enrollment  in 
rural  areas 

All  Ed  facilities  fully 
operational 

Decrease  in 
crime/vandalism  in 
and  around  schools 

Schools  develop  PE 
curriculum 

Higher  math  and 
science  test 

scores 

Schools  open  full  time 

Education  facilities  not 
in  use 

Math/science 
activities  develop 
(fairs,  competitions, 
etc.) 

Decrease  in 
preventable 
illnesses  in 
children 

Schools  fully  staffed 

Increase 

attendance  across 
the  board 

Schools  at  full 
capacity  for  students 

Increase  in 
enrollment 

Looking  at  the  Education  Infrastructure  system,  assessors  see  that  the  schools  are 
fully  staffed,  however,  assessors  note  that  there  are  two  indicators  present  for  system 
state  s 2 .  Likewise  the  schools  are  not  yet  to  full  capacity  for  students,  and  some  facilities 
are  either  not  yet  fully  operational  or  not  yet  open  full-time.  Therefore,  the  assessors 
declare  that  the  education  infrastructure  is  in  system  state  s 2 . 

4.3.9  Phases  VI  and  VII:  Objective  and  Campaign  Assessment. 

The  resulting  state  vector  from  the  indicator  analysis  is  (Students  ^infrastructure)  >  which 
is  denoted  (1,2).  The  planners  have  mapped  state  vector  (1,2)  to  approximately  60 
percent  on  the  progress  function.  Figure  4.21  lists  the  progress  function  at  the  18-month 
assessment.  From  the  assessment  at  12  months,  this  state  vector  represents  a  decrease  in 
progress  (Figure  4.22). 
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Figure  4.21  OCG  Progress  Function  at  18  Months 
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Figure  4.22  OCG  Actual  Progress  Line  for  Objective  1  at  18  Months 
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The  assessors  attribute  this  decrease  in  progress  to  the  fact  that  the  student  enrollment  had 
reached  pre-regime  levels  at  the  12-month  assessment  (reference  Appendix  C),  but  the 
number  of  students  enrolled  has  actually  decreased  since  then.  The  reason  for  the 
unexpected  decrease  is  unknown  and  warrants  further  investigation  by  the  assessors. 

4.4  Conclusion 

In  Chapter  4,  EBA  methodology  was  applied  to  two  scenarios.  The  first  scenario 
described  how  EBA  can  be  implemented  to  assess  an  air  campaign  involving  combat 
operations.  The  second  scenario  described  how  EBA  can  be  applied  to  a  nation-building 
campaign  involving  stability  operations.  Though  the  combat  operations  and  stability 
operations  have  very  different  aims,  the  EBA  methodology  presented  in  this  thesis  can 
provide  insight  for  each  case  into  the  accomplishment  of  planned  actions,  the 
achievement  of  desired  effects,  the  attainment  of  objectives,  and  the  progress  of  the 
overall  campaign.  Chapter  5,  will  discuss  the  advantages  of  this  EBA  methodology  as 
applied  to  these  two  scenarios  as  well  as  some  limitations  of  the  methodology  when  it  is 
put  into  practice. 
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5  Discussion 


5.1  Introduction 

Chapter  5  summarizes  the  results,  contributions,  and  recommendations  for  future 
research  of  the  Effects-Based  Assessment  (EBA)  methodology  presented  in  this  thesis. 
The  results  discussed  include  the  applicability  and  limitations  of  the  EBA  methodology  to 
combat  and  stability  operations.  The  primary  recommendation  of  this  thesis  for  future 
research  are  applications  of  stochastic  analysis  methods  to  the  problem  of  EBA. 

5.2  Results  of  the  Research 

This  thesis  set  out  to  formulate  an  EBA  construct  that  is  timely,  accurate, 
actionable,  efficient,  simple,  quantifiable,  adaptable,  and  most  importantly,  provides 
relevant  insight  to  the  CFACC  and  military  planners.  Chapter  4  demonstrated  how  the 
EBA  methodology  can  be  applied  to  both  combat  operations  and  stability  operations. 
However,  the  application  of  the  EBA  methodology  does  not  produce  equivalent  results 
for  both  types  of  operations. 

When  comparing  the  process  of  applying  the  EBA  methodology  to  OPERATION 
DENY  FORCE  (ODF)  and  the  process  of  applying  the  methodology  to  OPERATION 
CLOSE  GAP  (OCG),  tracking  the  effects  for  ODF  was  more  direct  than  tracking  the 
effects  for  OCG.  In  OCG,  the  desired  effects  of  the  stability  operations  lent  themselves 
to  being  tracked  more  easily  by  monitoring  the  systems  of  interest  directly.  In  ODF,  the 
systems  involved  included  softer,  human  systems,  which  are  inherently  harder  to  monitor. 
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The  desired  effects  in  ODF  therefore  had  to  be  largely  monitored  indirectly.  It  is  in  this 
example  that  the  methodology  proved  its  worth. 

The  main  contributor  to  the  method’s  applicability  is  the  development  of 
indicators  in  Phase  II.  When  the  system  state  can  be  measured  directly,  a  formal 
methodology  for  determining  system  state  indicators  is  not  necessarily  required.  One 
need  not  watch  for  “red  faced”  people  to  determine  the  outside  temperature  if  a 
thermometer  were  available.  The  same  holds  true  for  EBA.  If  a  system  can  be  measured 
directly,  there  is  no  need  to  go  through  the  effects-tree  analysis  described  in  Phase  II. 

The  use  of  direct  metrics  would  be  more  efficient  and  more  appropriate  than  proxy 
indicators  when  the  system  can  be  measured  directly  Additionally,  an  added  level  of 
uncertainty  accompanies  the  process  when  measuring  systems  indirectly.  For  example, 
“red  faced”  people  may  be  an  indicator  of  temperature,  but  it  could  also  be  an  indicator  of 
a  state  of  embarrassment. 

In  Phase  II,  additional  indicators  of  indirect  effects  are  developed  under  a  strict 
assumption  about  the  enemy:  the  enemy  will  continue  to  fight  by  all  means  available  to 
him  (reference  Section  2. 1 1).  Under  this  assumption,  assessors  can  consider  how 
individual  friendly  actions  affect  the  enemy;  these  individual  considerations  then  lead  to  a 
range  of  potential  system  reactions  that  describe  all  possible  states  of  the  system.  This 
range  or  enemy  reactions  is  vital  to  the  fidelity  of  effects  assessment  and  system  state 
definition  in  Phase  III. 

In  stability  operations,  however,  there  is  not  always  an  enemy  per  se.  In  the  OCG 
scenario  (reference  Section  4.3)  there  is  no  enemy  that  continues  to  fight  by  all  means 
available.  Therefore,  for  the  purposes  of  the  effects-tree  diagram,  the  assumption  about 
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the  enemy  is  violated.  This  violation  can  result  in  a  very  narrow  range  of  indirect  effects 
produced  by  friendly  actions,  which  in  turn  can  result  in  a  very  narrow  range  of 
indicators.  A  narrow  range  of  indicators  leads  to  a  less  robust  range  of  ordinal  system 
states  produced  in  Phase  III,  which  results  in  a  set  of  system  states  that  do  not  completely 
represent  the  system  as  it  transitions  as  a  result  of  friendly  actions.  Ultimately,  the 
differences  in  state  vectors  become  more  difficult  to  distinguish,  which  then  makes  the 
assessed  progress  of  the  objectives  less  insightful. 

In  summary,  the  EBA  methodology  presented  in  this  thesis  can  be  used  to  assess 
combat  operations  and  stability  operations,  however,  the  methodology  is  most  useful 
when  assessing  combat  operations  involving  systems  that  cannot  be  assessed  directly. 

5.3  Contributions  of  the  Research 

This  thesis  advances  the  application  of  EBA  by  defining  anticipated  states  of 
enemy  systems,  developing  indicators  to  determine  the  state  of  enemy  systems,  and 
applying  progress  functions  to  the  anticipated  and  desired  states  of  the  enemy  systems  in 
order  to  describe  progress  of  the  campaign  towards  attainment  of  the  commander’s 
objectives. 

Defining  effects  as  a  change  in  a  system  state  formulates  the  EBA  problem  in 
tenns  that  allow  for  the  application  of  more  robust  modeling  techniques.  Defining 
system  states  allows  planners  and  assessors  to  model  effects  ( model  in  the  general  sense) 
by  considering  a  system’s  transitions  from  an  initial  state  to  a  desired  state.  System  states 
offer  assessors  a  more  tangible  method  for  assessing  effects.  When  effects  are  described 
in  terms  of  system  states,  the  planners  consider  actions  to  induce  a  system  to  transition 
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into  a  desired  state.  Assessors  then  consider  evidence  of  the  system  state.  This  is  more 
robust  than  physical  and  functional  assessments,  because  it  explicitly  takes  into  account 
the  indirect  effects  influenced  by  the  system  states.  These  indirect  effects  are  at  the  heart 
of  the  operational  value  of  EBO. 

Similarly,  the  deliberate  and  methodical  development  of  indicators  contributes  to 
both  planning  and  assessing  effects-based  operations.  EBO  is  about  effects;  as  such  the 
EBO  process  should  include  a  method  wherein  planners  and  assessors  consider  the  direct 
and  indirect  effects  of  planned  and  executed  military  actions.  The  effects-tree  diagram 
provides  a  means  to  accomplish  this  consideration.  The  effects-tree  diagram  combines 
the  system  states  and  planned  friendly  actions  to  generate  a  list  of  indirect  effects.  These 
indirect  effects  aid  the  planners  by  helping  them  determine  potential  collateral  effects  of 
the  actions,  potential  unforeseen  enemy  reactions  which  can  be  mitigated  by  additional  or 
different  actions,  and  can  highlight  deficiencies  in  branch  plans  and  sequels.  Effects-tree 
diagrams  aid  assessors  by  leading  to  measurable  indicators  of  indirect  effects,  which  in 
turn  can  be  used  to  describe  enemy  system  states  that  represent  the  achievement  of 
desired  effects. 

5.4  Recommendations  for  Future  Research 

This  thesis  offers  two  main  suggestions  for  future  research:  methods  to  define 
system  states  directly,  prior  to  developing  indicators;  and  the  use  of  stochastic  modeling 
methods  to  determine  current  and  future  states  of  enemy  systems.  Phase  III  develops 
states  by  partitioning  as  set  of  indicators  that  describes  a  range  of  potential  enemy  system 
reactions.  Another  approach  is  to  determine  the  intennediate  states  along  with  the  initial 
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state  and  desired  state  of  the  system  before  developing  indicators.  This  could  be  done 
through  analysis  of  the  SoSA  model  or  by  a  method  similar  to  Gallagher’s  specifications 
for  a  precisely  defined  effect.  Additional  specifications  could  include  items  that  describe 
intennediate  states  witnessed  through  combat  modeling  and  systems-of-systems  analysis. 

Smith  describes  the  second  avenue  for  future  research,  which  involves  the 
development  of  collections  and  indications  algorithms  similar  to  indications  and  warning 
intelligence  used  during  the  Cold  War  era.  These  algorithms  were  developed  to  yield  an 
overall  probability  of  an  enemy  attack.  A  general  process  would  nominate  indicators, 
apply  weights  to  those  indicators  and  assemble  the  various  inputs  into  a  “coherent 
probabilistic  understanding”  of  the  nature  of  an  enemy’s  behavior  (Smith,  2002:382, 

398).  Stochastic  methods  exist  to  produce  this  coherent  probabilistic  understanding.  One 
such  method  is  the  use  of  Hidden  Markov  Models  (HMM).  HMMs  are  widely  used  in 
speech  recognition  and  the  comparing  of  protein  sequences.  HMMs  have  also  been 
applied  to  problems  of  conflict  detection,  failure  analysis,  medical  diagnosis,  knowledge 
assessment,  and  pattern  recognition. 

An  HMM  is  variation  of  a  Markov  chain  model.  HMMs  consist  of  a  set  of 
discrete  states  and  a  matrix  P  =  {  ptj }  of  transition  probabilities  for  a  system  moving  from 

state  i  to  state  j.  In  addition,  every  state  has  a  vector  of  observed  symbol  (or  indicator) 
probabilities,  I  =  {ij(k)}  that  corresponds  to  the  probability  that  the  system  will  produce 

a  symbol  of  type  k  when  the  system  is  in  state  j.  Furthermore,  the  states  cannot  be 
directly  observed  and  can  only  be  detennined  from  the  indicators,  ergo  the  term 
“hidden”  (Schrodt,  1997:12). 
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This  description  of  HMMs  aligns  well  with  the  Phase  V  method  of  using 
indicators  to  determine  system  states  presented  in  this  thesis.  To  apply  HMM  to  EBA, 
assessors  would  need  to  assign  the  state  transition  probabilities  and  the  indicator 
probabilities,  which  is  no  small  task.  The  benefit  would  be  that  HMMs  would  help 
answer  questions  such  as  “given  a  set  of  observed  indicators,  what  is  the  most  likely  state 
of  the  system?”  and  “what  state  is  the  system  most  likely  to  transition  to?”  Further 
research  into  the  applicability  of  HMMs  to  EBO  could  lead  to  more  insightful 
assessments,  which  may  include  a  methodology  for  predicting  future  states  of  enemy 
systems,  a  capability  highly  sought  after  by  planners  looking  days  and  weeks  ahead  of 
current  operations.  Rabiner  (1989)  and  Schrodt  (1997)  offer  a  baseline  for  HMMs; 
Falmagne  and  Doignon  (1988)  describe  a  similar  procedure  using  finite  Markov  chains 
for  assessing  system  states. 

5.5  Conclusion 

The  EBA  methodology  presented  in  this  thesis  takes  a  big  step  towards  complete 
operational  implementation  of  EBO.  As  demonstrated  in  Chapter  4,  the  seven-phase 
EBA  methodology  can  be  applied  to  combat  operations  as  well  as  stability  operations, 
though  it  is  suited  best  for  combat  operations  where  the  systems  of  interest  cannot  be 
assessed  directly.  The  contributions  of  the  EBA  construct  presented  here  include  the 
definition  of  enemy  system  states  to  be  tracked  as  the  systems  change  in  order  to 
determine  achievement  of  desired  effects;  the  development  of  indicators  of  system  states 
using  effects-tree  diagrams;  and  the  mapping  of  the  system  state  vectors  to  approximate 
values  of  objective  attainment  via  progress  functions.  Formulating  desired  effects  in 
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terms  of  system  state  transitions  and  using  indicators  to  determine  those  states  opens  two 
areas  for  future  research:  defining  the  system  states  directly  along  with  the  initial  state 
and  desired  end  state  of  the  system,  and  the  use  of  Hidden  Markov  Models  to  detennine 
the  most  likely  system  state  and  most  likely  future  state  given  a  set  of  observed 
indicators.  In  time,  the  EBA  paradigm  presented  here,  combined  with  further  research, 
doctrinal  development,  and  operational  experience,  will  increase  the  crucial  insight  to 
commanders  into  the  complex  effects  created  by  friendly  actions  on  the  battlespace  and 
the  enemy. 
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Appendix  A  Effects-Tree  Diagram  Example 


Figure  A.1  Effects-Tree  Diagram  Example  (reference  Section  3.3.2.2) 
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Appendix  B  OPERATION  DENY  FORCE  Data 


Appendix  B  presents  the  data  used  to  make  the  assessments  in  the  OPERATION 
DENY  FORCE  (ODF)  scenario  in  Chapter  4.  The  daily  cumulative  actions 
accomplishment  are  presented  here  for  campaign  days  D+l  through  D+3,  and  the 
observed  indicators  are  presented  for  campaign  days  D+l  through  D+4. 

B.l  ODF  at  D+l 

Figure  B.  1  illustrates  the  cumulative  actions  accomplishment  at  D+l .  The  mean 
accomplishment  for  all  actions  is  69  percent. 
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Figure  B.l  ODF  Actions  Accomplishment  at  D+1 
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It  is  anticipated  that  Desired  Effect  1 .2  (IADS  neutralized)  will  be  achieved  when  80 
percent  of  the  actions  taken  against  the  IADS  are  accomplished.  The  other  actions  must 
be  100  percent  accomplished. 

Tables  B.l  and  B.2  illustrate  the  indicators  observed  by  ISR  assets  at  D+l  for  the 
desired  effects  of  Objective  1.  For  Desired  Effects  1.1  and  1.2,  assessors  detennine  that 
the  occurrence  of  “No  SAFIRES”  puts  the  both  the  OLTO  airspace  and  the  IADS  in 
desired  state  se.  Therefore,  the  assessed  state  vector  for  Objective  1  is  (2,2). 


Table  B.l  ODF  Desired  Effect  1.1,  OLTO  Indicators  at  D+1 


m 

SOLTO 

S° 

s1 

5,e 

State  Description 
(Given  IADS  neutralized) 

OL  a/c  able  and  willing  to 
contest  coalition  forces. 

OL  a/c  contest  Coalition 
forces  indirectly. 

OL  experiences  complete  loss 
of  air  sovereignty  and 
coalition  forces  have  freedom 
of  access  for  follow-on 
persistence  forces. 

Indicators 

Increased  sortie  rates 

Civilian  a/c  engage  in 
unscheduled  or  unexpected 
flights 

No  SAFIRES 

OL  a/c  engage  Coalition 
forces 

Civilian  a/c  fly  unusual  flight 
patterns 

No  airborne  OL  a/c 

Mobile  SAMs  positioned 
near  high-priority  assets 

Coalition  forces  uncontested 
in  OL  airspace 

Table  B.2  ODF  Desired  Effect  1.2,  IADS  Indicators  at  D+1 


o'” 

‘’IADS 

S° 

s1 

State  Description 

Little  or  no  change  in  IADS 
capability  or  behavior. 

IADS  undergone  a 
significant  change. 

IADS  threat  to  coalition 
forces  neutralized. 

Indicators 

Guided  SAFIRES 

Decrease  in  SAFIRES 

No  SAFIRES 

Decreased  radar  emissions 

Ballistic  SAFIRES 

Decreased  comm,  emissions 

No  radar  emissions 

Tables  B.3  through  B.5  illustrate  the  indicators  observed  for  the  desired  effects  of 


Objective  2.  For  Desired  Effect  2.1  (Table  B.3),  the  only  indicator  observed  of  the  OL 
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leadership  reaction  is  a  protest  against  Coalition  actions  issued  through  a  third-party 
embassy.  OL  still  denies  any  WMD  activity.  Therefore,  assessors  detennine  the  OL 
leadership  to  be  in  the  initial  state  5°  because  the  disposition  of  the  OL  leadership  has  not 
changed.  For  Desired  Effect  2.2  (Table  B.4),  assessors  determine  the  WMD  R&D 
facilities  to  be  in  the  desired  state  se  due  to  the  explosions  at  the  R&D  facilities,  and 
because  the  power  is  still  out  at  the  facilities  while  it  has  returned  to  the  surrounding  area. 
Likewise,  assessors  determine  the  WMD  deployment  facilities  to  be  in  the  desired  state  se 
due  to  seeing  one-third  of  the  indicators  for  the  desired  state,  which  is  a  greater 
proportion  than  the  indicators  for  system  state  sl.  Therefore,  the  state  vector  for 
Objective  2  is  (0,1,2). 


Table  B.3  ODF  Desired  Effect  2.1,  OL  Leadership  Indicators  at  D+1 


m 

^OL 

s° 

sl 

s2 

State  Description 

Little  or  no  change  to 
system  function. 

System  functionality 
affected. 

Little  bit  better.. 

System  unable  to 
perform  mission. 

Indicators 

OL  launches  TBM  at 
Coalition  forces 

OL  civilian  a/c  used 
as  WMD  (flying 
irregular 
pattems/times) 

OL  emplaces 
civilians  I VO 

WMD  sites 

OL  issues  a  demarche 
against  US 

OL  launches  TBM  at 
neighboring  countries 

OL  publicizes 
civilian  consequences 
(blackouts  to 
hospitals/schools, 
casualties  etc.) 

Military  defenses 
relocated  to 

WMD  R&D 
facilities 

OL  initiates  diplomacy 
via  third  party 

Increased  terrorist 
attacks  in  US/Coalition 
allies 

OL  launches  TBM 
into  own  territory 

Increased  heavy 
equipment  traffic 
in/out  of  WMD 
facilities 

Terrorist  attacks  in 
neighboring  countries 
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Table  B.4  ODF  Desired  Effect  2.2,  WMD  R&D  Indicators  at  D+1 


m 

^  WMD  R&D 

0 

S' 

se 

State  Description 

Little  or  no  change  to  system 
function. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy-equipment 
traffic  to  and  from  facilities 

Local  blackout  around  R&D 
facilities 

Increased  security  at  WMD 
R&D  facilities 

Lights  stay  out  at  R&D 
facilities  while  power  returns 
to  surrounding  area 

Mobile  SAM  IVO  WMD 

R&D  facilities 

Decreased  traffic  to  facilities 

Explosions,  fires  at  R&D 
facilities 

Table  B.5  ODF  Desired  Effect  2.3,  WMD  Deployment  Indicators  at  D+1 


m 

^  WMD  Dep 

s° 

s1 

se 

State  Description 

Little  or  no  change  to  system 
function. 

System  functionality  affected. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy  equipment 
traffic  at  WMD  deployment 
sites 

Increased  cell  phone  use 

Fires  and  explosions  at  WMD 
deployment  sites 

Increased  traffic  out  of/ 
decreased  traffic  to  WMD 
deployment  sites 

Increased  radio  use 

OL  forces  evacuate  WMD 
deployment  sites* 

OL  forces  garrison  at  WMD 
deployment  sites* 

Increased  SATCOM  use 

OL  use  courier  to 
communicate* 

unmeasurable  indicator 


B.2  ODF  at  D+2 

Figure  B.2  illustrates  the  cumulative  actions  accomplishment  at  D+2.  The  mean 
accomplishment  for  all  actions  is  78  percent.  All  actions  associated  with  the  IADS  have 
achieved  above  70  percent,  as  have  all  other  actions.  The  actions  against  the  EP 
generators  feeding  the  WMD  R&D  facilities  have  been  100  percent  accomplished. 

Tables  B.6  and  B.7  illustrate  the  indicators  observed  by  ISR  assets  at  D+2  for  the 
desired  effects  of  Objective  1.  For  Desired  Effects  1.1,  assessors  observe  two  indicators 
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of  state  5°.  For  Desired  Effect  1.2,  assessors  observe  all  the  indicators  of  state  sl. 
Therefore,  the  assessed  state  vector  for  Objective  1  is  (0,1). 


,  Target 
Line 


1.00 


'  I  IADS  □  EP  for  WMD  R&D  □  WMD  deployment 


Figure  B.2  ODF  Actions  Accomplishment  at  D+2 


Table  B.6  ODF  Desired  Effect  1.1,  OLTO  Indicators  at  D+2 


m 

SOLTO 

S° 

s1 

5,e 

State  Description 
(Given  IADS  neutralized) 

OL  a/c  able  and  willing  to 
contest  coalition  forces. 

OL  a/c  contest  Coalition 
forces  indirectly. 

OL  experiences  complete  loss 
of  air  sovereignty  and 
coalition  forces  have  freedom 
of  access  for  follow-on 
persistence  forces. 

Indicators 

Increased  sortie  rates 

Civilian  a/c  engage  in 
unscheduled  or  unexpected 
flights 

No  SAFIRES 

OL  a/c  engage  Coalition 
forces 

Civilian  a/c  fly  unusual  flight 
patterns 

No  airborne  OL  a/c 

Mobile  SAMs  positioned 
near  high-priority  assets 

Coalition  forces  uncontested 
in  OL  airspace 
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Table  B.7  ODF  Desired  Effect  1.2,  IADS  Indicators  at  D+2 


m 

S\ADS, 

S° 

s1 

State  Description 

Little  or  no  change  in  IADS 
capability  or  behavior. 

IADS  undergone  a 
significant  change. 

IADS  threat  to  coalition 
forces  neutralized. 

Indicators 

Guided  SAFIRES 

Decrease  in  SAFIRES 

No  SAFIRES 

Decreased  radar  emissions 

Ballistic  SAFIRES 

Decreased  comm,  emissions 

No  radar  emissions 

Tables  B.8  through  B.10  illustrate  the  indicators  observed  for  the  desired  effects 
of  Objective  2.  For  Desired  Effect  2.1  (Table  B.8),  assessors  observe  two  indicators  of 
state  51  in  addition  to  the  OL  leadership  reaction  of  a  protest  against  Coalition  actions 
issued  through  a  third-party  embassy.  Though  OL  continues  to  deny  any  WMD  activity, 
the  use  of  civilians  as  a  deterrent  to  Coalition  actions  places  OL  in  system  state  .s1. 


Table  B.8  ODF  Desired  Effect  2.1,  OL  Leadership  Indicators  at  D+2 


m 

^OL 

S0 

i 

V 

2 

s 

State  Description 

Little  or  no  change  to 
system  function. 

System  functionality 
affected. 

Little  bit  better.. 

System  unable  to 
perform  mission. 

Indicators 

OL  launches  TBM  at 
Coalition  forces 

OL  civilian  a/c  used 
as  WMD  (flying 
irregular 
pattems/times) 

OL  emplaces 
civilians  IVO 

WMD  sites 

OL  issues  a  demarche 
against  US 

OL  launches  TBM  at 
neighboring  countries 

OL  publicizes 
civilian  consequences 
(blackouts  to 
hospitals/schools, 
casualties  etc.) 

Military  defenses 
relocated  to 

WMD  R&D 
facilities 

OL  initiates  diplomacy 
via  third  party 

Increased  terrorist 
attacks  in  US/Coalition 
allies 

OL  launches  TBM 
into  own  territory 

Increased  heavy 
equipment  traffic 
in/out  of  WMD 
facilities 

Terrorist  attacks  in 
neighboring  countries 

For  Desired  Effect  2.2  (Table  B.9),  the  electrical  power  indicators  remain,  so  the 


WMD  R&D  facilities  remain  in  the  desired  state  se.  For  Desired  Effect  2.3  (Table  B.  10), 
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assessors  observe  increased  traffic,  both  regular  and  heavy  equipment,  to  and  from  the 
WMD  deployment  sites.  Therefore,  the  state  vector  for  Objective  2  is  (1,1,0). 


Table  B.9  ODF  Desired  Effect  2.2,  WMD  R&D  Indicators  at  D+2 


m 

°  WMD  R&D 

0 

S' 

se 

State  Description 

Little  or  no  change  to  system 
function. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy-equipment 
traffic  to  and  from  facilities 

Local  blackout  around  R&D 
facilities 

Increased  security  at  WMD 
R&D  facilities 

Lights  stay  out  at  R&D 
facilities  while  power  returns 
to  surrounding  area 

Mobile  SAM  IVO  WMD 

R&D  facilities 

Decreased  traffic  to  facilities 

Explosions,  fires  at  R&D 
facilities 

Table  B.10  ODF  Desired  Effect  2.3,  WMD  Deployment  Indicators  at  D+2 


m 

^  WMD  Dep 

s° 

s1 

5,C 

State  Description 

Little  or  no  change  to  system 
function. 

System  functionality  affected. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy  equipment 
traffic  at  WMD  deployment 
sites 

Increased  cell  phone  use 

Fires  and  explosions  at  WMD 
deployment  sites 

Increased  traffic  out  of/ 
decreased  traffic  to  WMD 
deployment  sites 

Increased  radio  use 

OL  forces  evacuate  WMD 
deployment  sites* 

OL  forces  garrison  at  WMD 
deployment  sites* 

Increased  SATCOM  use 

OL  use  courier  to 
communicate* 

*  unmeasurable  indicator 


B.3  ODF  at  D+3 

Figure  B.3  illustrates  the  cumulative  actions  accomplishment  at  D+3.  The  mean 
accomplishment  for  all  actions  has  increased  to  8 1  percent.  All  actions  associated  with 
the  IADS  have  been  accomplished  above  the  required  80-percent  level.  However,  OL 
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has  reconstituted  some  of  the  EP  systems  and  the  POL  for  WMD  deployment,  which 
negates  some  of  the  actions  accomplishment  for  Objective  2. 


IADS  □  EPforWMD  R&D  □  WMD  deployment 


Figure  B.3  ODF  Actions  Accomplishment  at  D+3 


For  Objective  1,  Tables  B.l  1  and  B.12  illustrate  that  assessors  continue  observe 
no  SAFIRES,  however  OL  is  continuing  to  use  civilian  aircraft  to  challenge  Coalition 
forces  with  the  added  indicator  that  OL  military  aircraft  are  now  engaging  Coalition 
forces.  This  places  the  OLTO  airspace  in  state  .v1  and  the  IADS  in  state  ,s,c.  Therefore  the 
state  vector  for  Objective  1  is  (1,2). 
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Table  B.11  ODF  Desired  Effect  1.1,  OLTO  Indicators  at  D+3 


‘’OLTO 

S° 

s1 

State  Description 
(Given  IADS  neutralized) 

OL  a/c  able  and  willing  to 
contest  coalition  forces. 

OL  a/c  contest  Coalition 
forces  indirectly. 

OL  experiences  complete  loss 
of  air  sovereignty  and 
coalition  forces  have  freedom 
of  access  for  follow-on 
persistence  forces. 

Indicators 

Increased  sortie  rates 

Civilian  a/c  engage  in 
unscheduled  or  unexpected 
flights 

No  SAFIRES 

OL  a/c  engage  Coalition 
forces 

Civilian  a/c  fly  unusual  flight 
patterns 

No  airborne  OL  a/c 

Mobile  SAMs  positioned 
near  high-priority  assets 

Coalition  forces  uncontested 
in  OL  airspace 

Table  B.12  ODF  Desired  Effect  1.2,  IADS  Indicators  at  D+3 


o'" 

‘’IADS 

0 

5 

1 

5 

S* 

State  Description 

Little  or  no  change  in  IADS 
capability  or  behavior. 

IADS  undergone  a 
significant  change. 

IADS  threat  to  coalition 
forces  neutralized. 

Indicators 

Guided  SAFIRES 

Decrease  in  SAFIRES 

No  SAFIRES 

Decreased  radar  emissions 

Ballistic  SAFIRES 

Decreased  comm,  emissions 

No  radar  emissions 

In  Objective  2,  assessors  observe  the  indicator  that  OL  has  emplaced  large 
numbers  of  civilian  “protestors”  in  the  vicinity  of  suspected  WMD  sites.  This  lone 
indicator  places  the  OL  leadership  in  state  s  ,  as  seen  in  Table  B.  13.  For  Desired  Effect 
2.2  (Table  B.14),  assessors  again  observe  the  same  indicators  involving  the  electrical 
power  supply,  and  deem  the  WMD  R&D  facilities  in  the  desired  state  se.  For  Desired 
Effect  2.3  (Table  B.15),  assessors  observe  the  continuation  of  traffic  in  and  out  of  the 
WMD  deployment  sites  with  the  additional  indicators  of  increased  use  of  alternate 
communications.  The  assessors  determine  the  WMD  R&D  facilities  to  be  in  state  s1. 
Therefore,  the  state  vector  for  Objective  2  is  (2,1,1). 
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Table  B.13  ODF  Desired  Effect  2.1,  OL  Leadership  Indicators  at  D+3 


m 

^OL 

S'0 

i 

s 

2 

s 

State  Description 

Little  or  no  change  to 
system  function. 

System  functionality 
affected. 

Little  bit  better.. 

System  unable  to 
perform  mission. 

Indicators 

OL  launches  TBM  at 
Coalition  forces 

OL  civilian  a/c  used 
as  WMD  (flying 
irregular 
pattems/times) 

OL  emplaces 
civilians  IVO 

WMD  sites 

OL  issues  a  demarche 
against  US 

OL  launches  TBM  at 
neighboring  countries 

OL  publicizes 
civilian  consequences 
(blackouts  to 
hospitals/schools, 
casualties  etc.) 

Military  defenses 
relocated  to 

WMD  R&D 
facilities 

OL  initiates  diplomacy 
via  third  party 

Increased  terrorist 
attacks  in  US/Coalition 
allies 

OL  launches  TBM 
into  own  territory 

Increased  heavy 
equipment  traffic 
in/out  of  WMD 
facilities 

Terrorist  attacks  in 
neighboring  countries 

Table  B.14  ODF  Desired  Effect  2.2,  WMD  R&D  Indicators  at  D+3 


m 

°  WMD  R&D 

0 

S' 

se 

State  Description 

Little  or  no  change  to  system 
function. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy-equipment 
traffic  to  and  from  facilities 

Local  blackout  around  R&D 
facilities 

Increased  security  at  WMD 
R&D  facilities 

Lights  stay  out  at  R&D 
facilities  while  power  returns 
to  surrounding  area 

Mobile  SAM  IVO  WMD 

R&D  facilities 

Decreased  traffic  to  facilities 

Explosions,  fires  at  R&D 
facilities 

Table  B.15  ODF  Desired  Effect  2.3,  WMD  Deployment  Indicators  at  D+3 


m 

^  WMD  Dep 

s° 

s1 

State  Description 

Little  or  no  change  to  system 
function. 

System  functionality  affected. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy  equipment 
traffic  at  WMD  deployment 
sites 

Increased  cell  phone  use 

Fires  and  explosions  at  WMD 
deployment  sites 

Increased  traffic  out  of/ 
decreased  traffic  to  WMD 
deployment  sites 

Increased  radio  use 

OL  forces  evacuate  WMD 
deployment  sites* 

OL  forces  garrison  at  WMD 
deployment  sites* 

Increased  SATCOM  use 

OL  use  courier  to 
communicate* 

unmeasurable  indicator 
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B.4  ODF  at  D+4 


Tables  B.16  through  B.17  present  the  observed  indicators  for  ODF  at  D+4.  For 
Objective  1,  assessors  see  for  Desired  Effect  1.1  (Table  B.16)  that  the  OLTO  airspace 
continues  to  be  affected  by  civilian  aircraft  even  as  the  SAFIRE  threat  appears  to  be 
neutralized.  Assessors  detennine  the  OLTO  airspace  to  be  in  state  and  the  IADS 
(Table  B.17)  to  be  in  the  desired  state  se.  Therefore,  the  state  vector  for  Objective  1  is 
assessed  to  be  (1,2). 


Table  B.16  ODF  Desired  Effect  1.1,  OLTO  Indicators  at  D+4 


O' 

‘’OLTO 

S° 

s1 

State  Description 
(Given  IADS  neutralized) 

OL  a/c  able  and  willing  to 
contest  coalition  forces. 

OL  a/c  contest  Coalition 
forces  indirectly. 

OL  experiences  complete  loss 
of  air  sovereignty  and 
coalition  forces  have  freedom 
of  access  for  follow-on 
persistence  forces. 

Indicators 

Increased  sortie  rates 

Civilian  a/c  engage  in 
unscheduled  or  unexpected 
flights 

No  SAFIRES 

OL  a/c  engage  Coalition 
forces 

Civilian  a/c  fly  unusual  flight 
patterns 

No  airborne  OL  a/c 

Mobile  SAMs  positioned 
near  high-priority  assets 

Coalition  forces  uncontested 
in  OL  airspace 

Table  B.17  ODF  Desired  Effect  1.2,  IADS  Indicators  at  D+4 


m 

S\ADS, 

0 

5 

1 

5 

5e 

State  Description 

Little  or  no  change  in  IADS 
capability  or  behavior. 

IADS  undergone  a 
significant  change. 

IADS  threat  to  coalition 
forces  neutralized. 

Indicators 

Guided  SAFIRES 

Decrease  in  SAFIRES 

No  SAFIRES 

Decreased  radar  emissions 

Ballistic  SAFIRES 

Decreased  comm,  emissions 

No  radar  emissions 

In  Desired  Effect  2.1  (Table  B.18),  assessors  observe  OL  launching  TBMs  at 


neighboring  countries  and  within  OL  territory.  Assessors  also  observe  the  continued  use 
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of  civilian  aircraft  and  the  exploitation  of  civilians  through  propaganda.  Since  three  of 
these  indicators  belong  to  state  s\  assessors  detennine  the  OL  leadership  to  be  in  .s  '. 


Table  B.18  ODF  Desired  Effect  2.1,  OL  Leadership  Indicators  at  D+4 


m 

^OL 

S’0 

i 

s 

2 

s 

State  Description 

Little  or  no  change  to 
system  function. 

System  functionality 
affected. 

Little  bit  better.. 

System  unable  to 
perform  mission. 

Indicators 

OL  launches  TBM  at 
Coalition  forces 

OL  civilian  a/c  used 
as  WMD  (flying 
irregular 
pattems/times) 

OL  emplaces 
civilians  IVO 

WMD  sites 

OL  issues  a  demarche 
against  US 

OL  launches  TBM  at 
neighboring  countries 

OL  publicizes 
civilian  consequences 
(blackouts  to 
hospitals/ schools, 
casualties  etc.) 

Military  defenses 
relocated  to 

WMD  R&D 
facilities 

OL  initiates  diplomacy 
via  third  party 

Increased  terrorist 
attacks  in  US/Coalition 
allies 

OL  launches  TBM 
into  own  territory 

Increased  heavy 
equipment  traffic 
in/out  of  WMD 
facilities 

Terrorist  attacks  in 
neighboring  countries 

At  D+4,  assessors  see  a  significant  change  in  the  achievement  of  Desired  Effect 
2.2  (Table  B.19).  Due  to  OL  reconstitution  efforts,  assessors  now  see  the  return  of 
electrical  power  to  the  WMD  R&D  facilities.  Along  with  the  reconstitution  efforts, 
assessors  observe  increased  traffic  of  both  heavy  equipment  and  regular  traffic  at  the 
R&D  facilities.  These  indicators  place  the  WMD  R&D  facilities  in  the  initial  state  5°. 
However,  for  Desired  Effect  2.3  (Table  B.20)  assessors  observe  fires  and  explosions  at 
the  remaining  WMD  deployment  sites,  which  places  the  WMD  deployment  facilities  in 
the  desired  state  5°.  Therefore,  the  state  vector  for  Objective  2  is  assessed  to  be  (1,0,2). 
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Table  B.19  ODF  Desired  Effect  2.2,  WMD  R&D  Indicators  at  D+4 


m 

^  WMD  R&D 

0 

S' 

se 

State  Description 

Little  or  no  change  to  system 
function. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy-equipment 
traffic  to  and  from  facilities 

Local  blackout  around  R&D 
facilities 

Increased  security  at  WMD 
R&D  facilities 

Lights  stay  out  at  R&D 
facilities  while  power  returns 
to  surrounding  area 

Mobile  SAM  IVO  WMD 

R&D  facilities 

Decreased  traffic  to  facilities 

Explosions,  fires  at  R&D 
facilities 

Table  B.20  ODF  Desired  Effect  2.3,  WMD  Deployment  Indicators  at  D+4 


m 

^  WMD  Dep 

s° 

i 

S’ 

5,C 

State  Description 

Little  or  no  change  to  system 
function. 

System  functionality  affected. 

System  unable  to  perform 
mission. 

Indicators 

Increased  heavy  equipment 
traffic  at  WMD  deployment 
sites 

Increased  cell  phone  use 

Fires  and  explosions  at  WMD 
deployment  sites 

Increased  traffic  out  of/ 
decreased  traffic  to  WMD 
deployment  sites 

Increased  radio  use 

OL  forces  evacuate  WMD 
deployment  sites* 

OL  forces  garrison  at  WMD 
deployment  sites* 

Increased  SATCOM  use 

OL  use  courier  to 
communicate* 

*  unmeasurable  indicator 
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Appendix  C  OPERATION  CLOSE  GAP  Data 


Appendix  C  presents  the  data  used  to  make  the  assessments  in  the  OPERATION 
CLOSE  GAP  (OCG)  scenario  in  Chapter  4.  The  daily  cumulative  actions 
accomplishment  and  the  observed  indicators  are  presented  here  for  the  campaign  at  6 
months  and  the  campaign  at  12  months. 

C.l  OCG  at  6  Months 

Figure  C.  1  details  the  actions  accomplishment  for  OCG  at  six  months.  After  six 
months  of  actions,  only  the  action  to  develop  the  early  childhood  TV  series  is  completely 
accomplished.  The  mean  accomplishment  is  just  over  half-way  at  5 1  percent. 

Particularly  lagging  is  the  construction  of  new  schools,  providing  all  schools  with  potable 
water  and  sanitation  facilities,  and  the  enrollment  of  out-of-school  children  in  accelerated 
learning  programs,  all  of  which  have  only  been  just  over  20  percent  accomplished. 

Tables  C.l  and  C.2  illustrate  the  observed  indicators  at  six  months  for  OCG 
Objective  1  (Improve  quality  and  access  to  education).  For  Desired  Effect  1  (Table  C.l), 
assessors  observe  students  earning  higher  grades  and  an  increase  in  regular  attendance. 
However,  the  overall  enrollment  has  not  necessarily  increased.  Since  approximately  33 
percent  of  the  indicators  for  the  initial  state  5°  have  been  observed  compared  with  20 
percent  of  the  indicators  for  state  s1,  assessors  determine  that  the  student  enrollment  is  in 
5°.  For  Desired  Effect  2  (Table  C.2),  assessors  observe  indicators  for  states  5°,  s\  and  s2. 
Since  50  percent  of  the  indicators  for  state  s  have  been  observed,  which  is  greater  than 
the  other  two  states,  then  assessors  determine  the  Education  Infrastructure  to  be  in  state 
5  .  Therefore,  at  six  months,  the  state  vector  for  Objective  1  is  assessed  to  be  (1,2). 
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Percent  Accomplishment 


—  —  Target  Line 


6  mos 


Mean 


1.00  - 


0.80 


0.60 


0.40 


Rehab 
substandar 
d  schools 

Build  new 
schools 

Distribute 

sports 

equipment 

Provide 
schools 
with  potable 

Train  new 
teachers 
and 

Distribute 
desks  and 
chalkboard 

Distribute 
new  math 
and 

Establish 

model 

schools/cen 

Enroll  out- 
of-school 
children 

Distribute 

school 

supplies 

Develop  TV 
series 

6  mos 

0.55 

0.23 

0.52 

0.19 

0.70 

0.64 

0.58 

0.40 

0.22 

0.59 

1.00 

Mean 

0.51 

0.51 

0.51 

0.51 

0.51 

0.51 

0.51 

0.51 

0.51 

0.51 

0.51 

Actions 

Infrastructure  □  Students 


Figure  C.1  OCG  Actions  Accomplishment  at  6  Months 


Table  C.1  OCG  Desired  Effect  1,  Student  Enrollment  Indicators  at  6  Months 


m 

Students 

S° 

sl 

se 

State  Description 

Little  or  no  change 
to  enrollment  and 
attendance. 

Noticeable  increase 
in  enrollment  and 
attendance. 

System  at  desired 
levels.. 

Indicators 

Decrease  in  behavioral 
problems 

Development  of  sports, 
clubs,  and  after-school 
activities. 

Enrollment  at  desired 
level. 

Higher  standardized  test 
scores 

Waiting  lists  develop  for 
model  schools 

Attendance  at  desired 
level. 

Higher  grades 

Admini  strators/ teachers 
hold  workshops  and 
seminars  to  share  best 
practices 

Decrease  in  dropout  rates 

Increase  in  regular 
attendance 

Increased  enrollment 
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Table  C.2  OCG  Desired  Effect  2,  Education  Infrastructure  at  6  Months 


m 

s 

Infrastructure 

5° 

51 

s2 

State  Description 

Little  or  no  change  to 
system  capability. 

System  capability 
noticeably  increased. 

System  capability 

significantly 

increased. 

System  able  to 
perform  mission. 

Indicators 

Schools  hiring 
maintenance  staff 

Interscholastic 
athletic  competitions 
develop 

Increase  in 
attendance  in 
poor 

metropolitan 

areas 

Schools  used  as 
community  centers 

Increase  in  number  of 
teacher  and 
administrator 
applications 

Schools  initiate  sports 
programs 

Increase  in 
enrollment  in 
rural  areas 

All  Ed  facilities  fully 
operational 

Decrease  in 
crime/vandalism  in 
and  around  schools 

Schools  develop  PE 
curriculum 

Higher  math  and 
science  test 

scores 

Schools  open  full  time 

Education  facilities  not 
in  use 

Math/science 
activities  develop 
(fairs,  competitions, 
etc.) 

Decrease  in 
preventable 
illnesses  in 
children 

Schools  fully  staffed 

Increase 

attendance  across 
the  board 

Schools  at  full 
capacity  for  students 

Increase  in 
enrollment 

C.2  OCG  at  12  Months 

Figure  C.2  outlines  the  actions  accomplishment  at  12  months.  Assessors  note  that 
the  construction  of  new  schools  and  the  enrollment  of  out-of-school  students  into 
accelerated  programs  are  still  lagging  with  just  over  50  percent  accomplishment  each. 

The  mean  action  accomplishment  is  progressing,  however.  At  12  months,  the  mean 
accomplishment  is  70  percent. 

Tables  C.3  and  C.4  illustrate  the  observed  indicators  at  12  months  for  OCG 
Objective  1.  For  Desired  Effect  1  (Table  C.3),  assessors  observe  one  indicator  in  each 
state.  Since  the  indicator  of  the  desired  state  is  the  a  measure  of  the  system  of  interest 
itself  (the  student  enrollment),  assessors  determine  the  system  to  be  in  the  desired  state  se. 
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Percent  Accomplishment 


■  Target 
Line 


12  mos 


■  Infrastructure  □  Students 


Figure  C.2  OCG  Actions  Accomplishment  after  12  Months 


Table  C.3  OCG  Desired  Effect  1,  Student  Enrollment  Indicators  at  12  Months 


^  Students 

S° 

i 

s 

se 

State  Description 

Little  or  no  change 
to  enrollment  and 
attendance. 

Noticeable  increase 
in  enrollment  and 
attendance. 

System  at  desired 
levels.. 

Indicators 

Decrease  in  behavioral 
problems 

Development  of  sports, 
clubs,  and  after-school 
activities. 

Enrollment  at  desired 
level. 

Higher  standardized  test 
scores 

Waiting  lists  develop  for 
model  schools 

Attendance  at  desired 
level. 

Higher  grades 

Admini  strators/ teachers 
hold  workshops  and 
seminars  to  share  best 
practices 

Decrease  in  dropout  rates 

Increase  in  regular 
attendance 

Increased  enrollment 
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For  Desired  Effect  2  (Table  C.4),  assessors  observe  one  indicator  each  of  states  5° 
and  .s'1,  and  two  indicators  for  state  ,s2.  The  two  indicators  for  state  ,s2  account  for  33 
percent  of  state  ,v2’s  indicators,  while  one  indicator  for  states  5°  and  .s1  account  for  only  25 
percent  of  those  states’  indicators.  Assessors  therefore  determine  that  the  Education 
Infrastructure  is  in  state  .v2.  Therefore,  for  Objective  1,  assessors  determine  the  state 
vector  at  12  months  to  be  (1,2). 


Table  C.4  OCG  Desired  Effect  2,  Education  Infrastructure  at  12  Months 


m 

e 

Infrastructure 

s° 

51 

s2 

State  Description 

Little  or  no  change  to 
system  capability. 

System  capability 
noticeably  increased. 

System  capability 

significantly 

increased. 

System  able  to 
perform  mission. 

Indicators 

Schools  hiring 
maintenance  staff 

Interscholastic 
athletic  competitions 
develop 

Increase  in 
attendance  in 
poor 

metropolitan 

areas 

Schools  used  as 
community  centers 

Increase  in  number  of 
teacher  and 
administrator 
applications 

Schools  initiate  sports 
programs 

Increase  in 
enrollment  in 
rural  areas 

All  Ed  facilities  fully 
operational 

Decrease  in 
crime/vandalism  in 
and  around  schools 

Schools  develop  PE 
curriculum 

Higher  math  and 
science  test 

scores 

Schools  open  full  time 

Education  facilities  not 
in  use 

Math/science 
activities  develop 
(fairs,  competitions, 
etc.) 

Decrease  in 
preventable 
illnesses  in 
children 

Schools  folly  staffed 

Increase 

attendance  across 
the  board 

Schools  at  full 
capacity  for  students 

Increase  in 
enrollment 
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